Quality of service

The network between office locations and the data center or cloud must transport a multitude of applications and data, including high quality video or real-time voice. Bandwidth sensitive applications stretch the network’s capabilities and resources. Citrix SD-WAN provides guaranteed, secure, measurable, and predictable network services. This is achieved by managing the delay, jitter, bandwidth, and packet loss on the network.

The Citrix SD-WAN solution includes a sophisticated application Quality-of-Service (QoS) engine that accesses the application traffic and prioritizes critical applications. It also understands the requirements for WAN network quality, and picks a network path based on the quality characteristics in real time.

The topics in the following sections discuss QoS classes, IP rules, application QoS rules, and other components that are required to define application QoS.

From SD-WAN 11.5 release onwards, QoS features are configurable through Citrix SD-WAN Orchestrator service. For more information, see Quality of Service.

Classes

The Citrix SD-WAN configuration provides a default set of application and IP/Port based QoS policies that are applied to all traffic going over Virtual Paths. These settings can be customized to fit the deployment needs.

Classes are useful to prioritize the traffic. Application and IP/Port based QoS policies classify traffic and put it into appropriate classes specified in the configuration.

Citrix SD-WAN Orchestrator service supports 13 classes. For more information, see Classes.

The following are the different types of classes:

• Real-time: Used for low latency, low bandwidth, time-sensitive traffic. Real-time applications are time sensitive but don’t really need high bandwidth (for example voice over IP). Real-time applications are sensitive to latency and jitter, but can tolerate some loss.

• Interactive: Used for interactive traffic with low to medium latency requirements and low to medium bandwidth requirements. The interaction is typically between a client and a server. The communication might not need high bandwidth but is sensitive to loss and latency.

• Bulk: Used for high bandwidth traffic and applications that can tolerate high latency. Applications that handle file transfer and need high bandwidth are categorized as bulk class. These applications involve little human interference and are mostly handled by the systems themselves.

Bandwidth sharing among classes

Bandwidth is shared among classes as follows:

• Real-time: Traffic hitting real-time classes are guaranteed to have low latency and bandwidth is capped to the class share when there is competing traffic.

• Interactive: Traffic hitting the interactive classes get remaining bandwidth after serving real-time traffic and the available bandwidth is fair shared among the interactive classes.

• Bulk: Bulk is best effort. Bandwidth left over after serving real-time and interactive traffic is given to bulk classes on a fair share basis. Bulk traffic can starve if real-time and interactive traffic utilizes all the available bandwidth.

Note

Any class can use all available bandwidth when there is no contention.

The following example explains the bandwidth distribution based on the class configuration:

Consider there is an aggregated bandwidth of 10 Mbps over Virtual Path. If the class configuration is

• Real-time: 30%
• Interactive High: 40%
• Interactive Medium: 20%
• Interactive Low: 10%
• Bulk: 100%

The bandwidth distribution outcome is:

• Real-time traffic gets 30% of 10Mbs (3 Mbps) based on the need. If it needs less than 10%, then the rest of the bandwidth is made available to the other classes.

• Interactive classes share the remaining bandwidth on fair share basis (4 Mbps: 2 Mbps: 1 Mbps).

• Anything leftover when real-time, interactive traffic is not fully using their shares is given to the Bulk class.

Rules by IP address and port number

Rules by IP address and port number feature helps you to create rules for your network and take certain Quality of Service (QoS) decisions based on the rules. You can create custom rules for your network. For example, you can create a rule as – If source IP address is 172.186.30.74 and destination IP address is 172.186.10.89, set Transmit mode as Persistent Path and LAN to WAN Class as 10(realtime_class)”.

You can create rules locally at a site level or at the global level. If more than one site requires the same rule, you can create a template for rules globally under Global > Virtual Path Default Sets > Rules. The template can then be attached to the sites where the rules need to be applied. Even if a site is associated with the globally created rule template, you can create site specific rules. In such cases, site specific rules take precedence and override the globally created rule template.

From Citrix SD-WAN 11.5 release onwards, you can create IP rules using Citrix SD-WAN Orchestrator service. For more information, see IP rules.

Verify rules

Navigate to Monitoring > Flows. Select Flow Type field located in the Select Flows section at the top of the Flows page. Next to the Flow Type field there is a row of check boxes for selecting the flow information you want to view. Verify if the flow information is according to the configured rules.

Example: The rule “If source IP address is 172.186.30.74 and destination IP address is 172.186.10.89, set Transmit mode as Persistent Path” shows the following Flows Data.

Navigate to Monitoring > Statistics and verify the configured rules.

Rules by application name

The Application classification feature allows the Citrix SD-WAN appliance to parse incoming traffic and classify them as belonging to a particular application or application family. This classification allows us to enhance the QoS of individual application or application families by creating and applying application rules.

You can filter traffic flows based on application, application family, or application object match-types and apply application rules to them. he application rules are similar to Internet Protocol (IP) rules. For information on IP rules see, Rules by IP Address and Port Number.

For every application rule, you can specify the mode of transmission. The following are the available transmit modes:

• Load Balance Path: Application traffic for the flow is balanced across multiple paths. Traffic is sent through the best path until that path is used. The remaining packets are sent through the next best path.
• Persistent Path: Application traffic remains on the same path until the path is no longer available.
• Duplicate Path: Application traffic is duplicated across multiple paths, increasing reliability.

The application rules are associated to classes. For information on classes, see Customizing Classes.

By default, the following five pre-defined application rules are available for Citrix ICA applications:

How application rules are applied?

In the SD-WAN network, when the incoming packets reach the SD-WAN appliance, the initial few packets do not undergo DPI classification. At this point, the IP rule attributes such as Class, TCP termination are applied to the packets. After DPI classification, the application rule attributes such as Class, transmit mode override the IP rule attributes.

The IP rules have more number of attributes as compared to the application rules. The application rule overrides only a few IP rule attributes, the rest of the IP rule attributes remain processed on the packets.

For example, consider you have specified an application rule for a webmail application such as Google Mail that uses the SMTP protocol. The IP rule set for SMTP protocol is applied initially before DPI classification. After parsing the packets and classifying it as belonging to Google Mail application, the application rule specified for the Google Mail application is applied.

To create application rules using Citrix SD-WAN Orchestrator, see Application rules.

To confirm if application rules are applied to traffic flow, navigate to Monitoring > Flows.

Make a note of the app rule id and check if the class type and transmission mode are as per your rule configuration.

You can monitor the application QoS such as no of packets / bytes uploaded, downloaded, or dropped at each site by navigating to Monitoring > Statistics > Application QoS.

The Num parameter indicates the app rule id. Check for the app rule id obtained from the flow.

Creating custom applications

You can use application objects to define custom applications based on the following match types:

• IP protocol
• Application name
• Application family

The DPI classifier analyzes the incoming packets and classifies it as applications based on the specified match criteria. You can use these classified custom applications in QoS, firewall, and application routing.

Tip

You can specify one or more match types.

Application classification

The Citrix SD-WAN appliances perform deep packet inspection (DPI) to identify and classify applications using the following techniques:

• DPI library classification
• Citrix-proprietary Independent Computing Architecture (ICA) classification
• Application vendor APIs (for example Microsoft REST APIs for Office 365)
• Domain name based application classification

DPI library classification

The Deep Packet Inspection (DPI) library recognizes thousands of commercial applications. It enables real-time discovery and classification of applications. Using the DPI technology, the SD-WAN appliance analyses the incoming packets and classifies the traffic as belonging to a particular application or application family. Application classification for each connection takes a few packets.

To enable DPI library classification on Citrix SD-WAN Orchestrator service, see DPI library classification.

ICA classification

Citrix SD-WAN appliances can also identify and classify Citrix HDX traffic for virtual apps and desktops. Citrix SD-WAN recognizes the following variations of the ICA protocol:

• ICA
• ICA-CGP
• Single Stream ICA (SSI)
• Multi-Stream ICA (MSI)
• ICA over TCP
• ICA over UDP/EDT
• ICA over non-standard ports (including Multi-Port ICA)
• HDX Adaptive Transport
• ICA over WebSocket (used by HTML5 Receiver)

Note

Classification of ICA traffic delivered over SSL/TLS or DTLS is not supported in SD-WAN Standard Edition.

Classification of network traffic is done during initial connections or flow establishment. Therefore, pre-existing connections are not classified as ICA. Classification of connections is also lost when the connection table is cleared manually.

Framehawk traffic and Audio-over-UDP/RTP are not classified as HDX applications. They are reported as either “UDP” or “Unknown Protocol.”

Since release 10 version 1, the SD-WAN appliance can differentiate each ICA data stream in multi-stream ICA even in a single-port configuration. Each ICA stream is classified as a separate application with its own default QoS class for prioritization.

• For Multi-Stream ICA functionality to work properly, you must have SD-WAN Standard Edition 10.1 or above.

• For HDX user based reports to be shown on SDWAN-Center, you must have SD-WAN Standard Edition 11.0 or above.

Minimum software requirements for HDX information virtual channel:

• A Current Release of Citrix Virtual Apps and Desktops (formerly XenApp and XenDesktop), since the prerequisite functionality was introduced in XenApp and XenDesktop 7.17 and is not included in the 7.15 Long-Term Service Release.

• A version of the Citrix Workspace app (or its predecessor, Citrix Receiver) that supports multi-stream ICA and the HDX Insights information virtual channel, CTXNSAP. Look for HDX Insight with NSAP VC and Multiport/Multi-stream ICA in the Citrix Workspace app Feature Matrix. See the currently supported release versions at HDX Insights.

• From 11.2 release onwards, packet duplication is now enabled by default for HDX real-time traffic when multi-stream ICA is in use.

Once classified, the ICA application can be used in application rules and to view application statistics similar to other classified applications.

There are five default application rules for ICA applications one each for the following priority tags:

• Independent Computing Architecture (Citrix)(ICA)
• ICA Real-time (ica_priority_0)
• ICA Interactive (ica_priority_1)
• ICA Bulk-Transfer (ica_prority_2)
• ICA Background(ica_priority_3)

For more information, see Rules by Application Name

If you are running a combination of software that does not support Multi-Stream ICA over a single port, then to perform QoS you must configure multiple ports, one for each ICA stream. To classify HDX on non-standard ports as configured in XA/XD server policy, you must add those ports in ICA port configurations. Also, to match traffic on those ports to valid IP rules, you must update the ICA IP rules.

In the ICA IP and port list you can specify non-standard ports used in XA/XD policy to process for HDX classification. IP address is used to further restrict the ports to a specific destination. Use ‘*’ for port destined to any IP address. IP address with combination of SSL port is also used to indicate that the traffic is likely ICA even though the traffic is not finally classified as ICA. This indication is used to send L4 AppFlow records to support multi-hop reports in Citrix Application Delivery Management.

To enable ICA based classification on Citrix SD-WAN Orchestrator service, see ICA classification.

Application vendor API based classification

Citrix SD-WAN supports the following application vendor API based classification:

• Office 365. For more information, see Office 365 optimization.

• Citrix Cloud and Citrix Gateway service. For more information, see Gateway Service Optimization.

Domain name based application classification

The DPI classification engine is enhanced to classify applications based on the domain name and patterns. After the DNS forwarder intercepts and parses the DNS requests, the DPI engine uses the IP classifier to perform first packet classification. Further DPI library and ICA classification are done and the domain name based application ID is appended.

The Domain name based application feature allows you to group several domain names and treat it as a single application. Making it easier to apply firewall, application steering, QoS, and other rules. A maximum of 64 domain name based applications can be configured.

To define domain name based applications on Citrix SD-WAN Orchestrator service, see Domain name based application classification.

Note

• From 11.4.2 release onwards, the Domain name-based applications support configurable ports and protocol in Citrix SD-WAN Orchestrator service. For more information, see Domains and applications.

• From Citrix SD-WAN 11.5.0 release onwards, AAAA records are supported on Citrix SD-WAN Orchestrator service.

Limitations

• If there are no DNS request/response corresponding to a domain name based application, the DPI engine does not classify the domain name based application and hence does not apply the application rules corresponding to the domain name based application.
• If an Application Object is created such that the port range includes port 80 and/or port 443, with a specific IP address match type that corresponds to a domain name based application, the DPI engine does not classify the domain name based application.
• If explicit web proxies are configured, you have to add all the domain name patterns to the PAC file, to ensure that the DNS response does not always return the same IP address.
• The domain name based application classifications are reset on configuration upgrade. Reclassification happens based on pre 11.0.2 release classification techniques such as DPI library classification, ICA classification and Vendor application APIs based classification.
• The application signatures learned (destination IP addresses) by domain name based application classification are reset on configuration update.
• Only the standard DNS queries and their responses are processed.
• DNS response records split over multiple packets are not processed. Only DNS responses in a single packet are processed.
• DNS over TCP is not supported.
• Only top-level domains are supported as domain name patterns.

Classifying encrypted traffic

Citrix SD-WAN appliance detects and reports encrypted traffic, as part of application reporting, in the following two methods:

• For HTTPS traffic, the DPI engine inspects the SSL certificate to read the common name, which carries the name of the service (for example - Facebook, Twitter). Depending on the application architecture only one certificate might be used for several service types (for example - email, news, and so on). If different services use different certificates, the DPI engine would be able to differentiate between services.
• For applications that use their own encryption protocol, the DPI engine looks for binary patterns in the flows for instance in case of Skype the DPI engine looks for a binary pattern inside the certificate and determines the application.

Application Objects

Application objects enable you to group different types of match criteria into a single object that can be used in firewall policies and application steering. IP Protocol, Application, and Application Family are the available match types.

The following features use the application object as a match type:

• Application Routes

• Firewall policy

• Application QoS Rules

• Application QoE

Using Application Classification with a Firewall

The classification of traffic as applications, application families or domain names enables you to use the application, application families, and application objects as match types to filter traffic and apply firewall policy and rules. It applies for all Pre, Post, and local policies. For more information about firewall, see Stateful Firewall and NAT Support.

–>

Viewing Application Classification

After enabling application classification, you can view the application name and application family details in the following reports:

• Firewall connection Statistics

• Flows information

• Application statistics

Firewall connection statistics

Navigate to Monitoring > Firewall. Under Connections section, the Application and Family columns list the applications and its associated family.

If you do not enable application classification, the Application and Family columns do not show any data.

Flows Information

Navigate to Monitoring > Flows. Under Flows Data section, the Application column lists the application details.

Application statistics

Navigate to Monitoring > Statistics. Under Application Statistics section, the Application column lists the application details.

Troubleshooting

After enabling application classification, you can view the reports under the Monitoring section and ensure that they show application details. For more information, see Viewing Application Classification.

If there is any unexpected behavior, collect the STS diagnostics bundle while the issue is being observed, and share it with the Citrix Support team.

The STS bundle can be created and downloaded using Configuration > System Maintenance > Diagnostics > Diagnostic Information.

QoS fairness (RED)

The QoS fairness feature improves the fairness of multiple virtual path flows by using QoS classes and Random Early Detection (RED). A virtual path can be assigned to one of the 16 different classes. A class can be one of three basic types:

• Realtime classes serve traffic flows that demand prompt service up to a certain bandwidth limit. Low latency is preferred over aggregate throughput.
• Interactive classes have lower priority than realtime but have absolute priority over bulk traffic.
• Bulk classes get what is left over from realtime and interactive classes, because latency is less important for bulk traffic.

Users specify different bandwidth requirements for different classes, which enable the virtual path scheduler to arbitrate competing bandwidth requests from multiple classes of the same type. The scheduler uses the Hierarchical Fair Service Curve (HFSC) algorithm to achieve fairness among the classes.

HFSC services classes in first-in, first-out (FIFO) order. Before scheduling packets, Citrix SD-WAN examines the amount of traffic pending for the packets class. When excessive traffic is pending, the packets are dropped instead of being put into the queue (tail dropping).

Why does TCP cause queuing?

TCP cannot control how quickly the network can transmit data. To control bandwidth, TCP implements the concept of a bandwidth window, which is the amount of unacknowledged traffic that it allows in the network. It initially starts with a small window and doubles the size of that window whenever acknowledgments are received. This is called the slow start or exponential growth phase.

TCP identifies network congestion by detecting dropped packets. If the TCP stack sends a burst of packets that introduce a 250 ms delay, TCP does not detect congestion if none of the packets are discarded, so it continues to increase the size of the window. It might continue to do so until the wait time reaches 600–800 ms.

When TCP is not in the slow start mode, it reduces the bandwidth by half when packet loss is detected, and increases the allowed bandwidth by one packet for each acknowledgment received. TCP therefore alternates between putting upward pressure on the bandwidth and backing off. Unfortunately, if the wait time reaches 800 ms by the time packet loss is detected, the bandwidth reduction causes a transmission delay.

Impact on QoS fairness

When TCP transmission delay occurs, providing any kind of fairness guarantee within a virtual-path class is difficult. The virtual path scheduler must apply tail-drop behavior to avoid holding enormous amounts of traffic. The nature of TCP connections is such that a small number of traffic flows to fill the virtual path, making it difficult for a new TCP connection to achieve a fair share of the bandwidth. Sharing bandwidth fairly requires making sure that bandwidth is available for new packets to be transmitted.

Random Early Detection

Random Early Detection (RED) prevents traffic queues from filling up and causing tail-drop actions. It prevents needless queuing by the virtual path scheduler, without affecting the throughput that a TCP connection can achieve.

For information on how to use and enable RED, see How to use RED.

MPLS queues

This feature simplifies creating SD-WAN configurations when adding a Multiprotocol Layer Switching (MPLS) WAN Link. Previously, each MPLS queue required one WAN Link to be created. Each WAN Link required a unique Virtual IP Address (VIP) to create the WAN Link and a unique Differentiated Services Code Point (DSCP) tag corresponding to the provider’s queuing scheme. After defining a WAN Link for each MPLS queue, the Intranet Service to map to a specific queue is defined.

Currently, a new MPLS specific WAN Link definition (that is, Access Type) is available. When a new Private MPLS Access Type is selected, you can define the MPLS queues associated with the WAN Link. This allows a single VIP with multiple DSCP tags that correspond to the provider’s queuing implementation for the MPLS WAN Link. This maps the Intranet Service to multiple MPLS Queues on a single MPLS WAN Link. For information on how to configure MPLS using Citrix SD-WAN Orchestrator service, see MPLS queues.

Note

If you have existing MPLS configurations and would like to implement the Private MPLS Access Type, contact Citrix Support for assistance.

Assign autopath group to virtual path-WAN Link

The Autopath Group defined is the same for the MCN and Client appliance. This allows the system to build the Paths automatically. At the MCN site, you can also expand the WAN Link associated with the virtual path.

View permitted rate and congestion for WAN links

The SD-WAN web interface now allows you to view the permitted rate for WAN Links and WAN Link Usages and whether a WAN Link, Path, or Virtual Path is in congested state. In the previous releases, this information was only available in SD-WAN log files and through the CLI. These options are now available in the web interface to help with troubleshooting.

View permitted rate

Permitted Rate is the amount of bandwidth that a particular WAN Link, Virtual Path Service, Intranet Service, or Internet Service is permitted to use at a given point in time. The permitted rate for a WAN Link is static, and is defined explicitly in the SD-WAN configuration. The permitted rate for a Virtual Path Service, Intranet Service, or Internet Service will fluctuate over time, in response to congestion, user demand, and Fair Shares, but will always be greater than or equal to the Minimum Reserved Bandwidth for the Service.

Monitor WAN link

Go to Monitor > Statistics, and select WAN Link from the Show drop-down list.

Go to Monitor > Statistics, and select WAN Link Usage from the Show drop-down list.

Monitor MPLS queues

Go to Monitor > Statistics, and select MPLS Queues from the Show drop-down list.

Troubleshooting MPLS queues

To check the status of MPLS queues, navigate to Monitor > Statistics and select Paths (summary) from the Show drop-down list. In the following example, the path from MPLS queue “q1” to “q3” is in DEAD state and shown in red. The path from MPLS queue “q1” to “q5” is in GOOD state and shown in green.

For detailed information on paths, select Paths (Detailed) from the Show drop-down list. The information on paths such as reason for the state, duration, source port, destination port, MTU are available

In the following example, the path from MPLS queue “q1” to “q3” is in DEAD state and the reason is PEER. The path from MPLS queue “q3” to “q1” is dead and the reason is SILENCE. The following table provides the list if available reasons and its descriptions.

To check the access interface and IP address associated with the MPLS queues, select Access Interfaces from the Show drop-down list.

You can download the log files for further troubleshooting. Navigate to Configuration > Logging/Monitoring and select SDWAN_paths.log or SDWAN_common.log from the Log Options tab.