Architecture
Internally, the SD-WAN 4000/5000 appliance contains several virtual machines:
- A Xen hypervisor
- A NetScaler instance
- At least two accelerator instances
- A management server instance that manages the GUI and other tasks
- Internal networking
Figure 2. SD-WAN 4100/5100 virtual machines, internal networks, and external port usage (inline deployment shown)
No WAN traffic enters or leaves the accelerators except as configured in the NetScaler instance. When the appliance is first used, the Provisioning Wizard sets up an initial configuration that provides communication and load balancing between the NetScaler instance and the accelerators.
The management service is the management configuration interface for the appliance, and provides access to key operating and monitoring elements of the appliance. The management service displays SD-WAN parameters as if they were from a single accelerator, and all changes made through this interface are applied to all the accelerator instances.
The Xen hypervisor hosts all the virtual machines. The hypervisor is not user-configurable and should not be accessed except at the request of Citrix.
Internal and External Networks
The external network interfaces are divided into two categories: traffic interfaces and management interfaces.
Traffic Interfaces—The traffic interfaces include all the network interfaces except ports 0/1 and 0/2, which are used only for management. Acceleration takes place only on the traffic interfaces.
Note: You must keep the traffic interfaces isolated from the management interface to prevent ARP flapping and other problems. This isolation can be achieved physically or by tagging management interface and traffic interface packets with different VLANs.
Management subnet—The virtual machines connect directly to the external management subnet, with different IP addresses for the management service, NetScaler instance, and XenServer.
Note: You must keep the traffic interfaces isolated from the management interface to prevent ARP flapping and other problems. This isolation can be achieved physically or by tagging management interface and traffic interface packets with different VLANs.
Private Internal traffic subnet—The accelerators’ accelerated ports are connected to the NetScaler instance internally in a one-arm mode, using an internal traffic subnet. There is no direct connection between the instances’ accelerated ports and the appliance’s external ports. All accelerated traffic to the accelerators is controlled by the NetScaler instance.
Since this internal subnet is not accessible from outside the appliance, it uses non-routable subnets in the 169.254.0.0/16 range. The NetScaler instance provides NAT for features that require routable access to the accelerator. Only the following two features of the accelerators require IP addresses that can be reached from the outside world:
- The signaling IP address, used for secure peering and the SD-WAN Plugin.
- IP addresses, used for communication with the router when the WCCP protocol is used.
In both cases, the number of externally visible IP addresses is independent of the number of accelerators the appliance has.
The internal traffic subnet requires two IP addresses per accelerator, plus an address for the NetScaler, plus one or two WCCP VIP addresses if WCCP is used. Since the internal network is private, it has an abundance of address space for these tasks.
Data Flow on the Private Traffic Subnet—The one-arm connection between the NetScaler instance and the accelerators uses the SD-WAN virtual inline mode, in which the NetScaler instance routes packets to the accelerators and the accelerators route them back to the NetScaler instance. Traffic flow over this internal traffic subnet is identical regardless of whether the mode visible to the outside world (on the external interfaces) is inline, virtual inline, or WCCP.
This traffic requires the SD-WAN “Return to Ethernet Sender” option, and the NetScaler MAC Address Forwarding and Use Subnet IP options, which are enabled by the Provisioning Wizard.
Deployment Mode Summary: The differences between WCCP mode, inline mode, and virtual inline mode can be summarized as follows:
- WCCP mode is a one-arm configuration. The accelerators establish WCCP control channels with the router. In WCCP mode, only one or two accelerators manage the WCCP control channel on behalf of all the accelerators. Data traffic is load-balanced across all the accelerators. When GRE encapsulation is used, the NetScaler instance performs GRE encapsulation/decapsulation on the data stream between itself and the router, allowing the data between the NetScaler and the accelerators to use a decapsulated, Level-2 configuration.
- Inline mode operates much the same as WCCP mode internally, but externally the appliance emulates a bridge, and no WCCP control channel is established. A packet that enters the appliance on one bridge port exits through the other bridge port. SD-WAN 4000 and 5000 appliances have multiple bridges to support multiple inline links.
- In virtual inline mode (used when WCCP and inline modes are not feasible), the appliance is deployed in a one-arm configuration, much like WCCP, but without the WCCP control channel. Traffic is sent to the appliance from the router, using policy-based routing (PBR) rules. The appliance processes the traffic and returns it to the router.
Figure 3. WCCP and virtual inline cabling
See SD-WAN 4100/5100 virtual machines, internal networks, and external port usage for a diagram of port usage on SD-WAN 4100/5100 appliances. Traffic ports are arranged as a set of accelerated bridges, while the management ports are independent. Typically only one management port is used.
Figure 4. Inline cabling
Accelerated Bridges
SD-WAN 4100/5100 appliances have multiple accelerated bridges. Different models have different numbers and types of bridge ports. The two ports making up such a bridge are called an “accelerated pair.” All current models include a built-in network bypass function. (Some older SD-WAN 4100-500 and 4100–1000 units do not include network bypass). The network bypass function (also called “fail to wire”) connects pairs of ports together if the appliance fails as a result of either power loss or software failure (as determined by an internal watchdog timer).
Inline deployment. The bypass function allows SD-WAN 4100/5100 to be deployed in line with your WAN, typically between your LAN and your WAN router, without introducing a point of network failure.
The accelerated bridges support either 1 Gbps or 10 Gbps data rates. Ethernet and SFP+ interfaces are supported, depending on model.
One-arm deployment. One-arm deployments are also supported, using WCCP or virtual inline modes. With such deployments, an SD-WAN 4000/5000 traffic port is connected directly to a port on the WAN router. The other port on the bridged pair is left unconnected.
Performance considerations. Inline deployments provide higher performance than the one-arm deployments, because the use of two ports instead of one doubles the peak throughput of the interfaces.
Peak throughput is important with SD-WAN 4100/5100 appliances, because the compressor provides acceleration in proportion to the compression ratio. That is, a connection that achieves 100:1 compression transfers data 100 times faster than an uncompressed connection, as long as the rest of the network path can keep up.
For example, take a datacenter with a 500 Mbps WAN link and a 1 Gbps LAN. The small 2:1 speed ratio between the WAN and LAN allows compression to provide only a 2x speedup on a whole-link basis, because there is no way to get data onto or off of the LAN at speeds above 1 Gbps. A 10 Gbps LAN, which allows a tenfold increase in peak data rates, is recommended for use with SD-WAN 4100/5100 deployments.
When an SD-WAN 4100/5100 appliance is deployed in a one-arm mode, the peak transfer rate is cut in half. An SD-WAN 4100/5100 in one-arm mode, connected to the router with a 1 Gbps LAN interface, saturates this interface when the WAN is running at full speed in both directions. For good performance, SD-WAN 4100/5100 must have a LAN interface that is much faster than the WAN. When the appliance is connected directly to the router in a one-arm mode, use a 10 Gbps router port.
Note
The 10 Gbps ports support 10 Gbps only. They do not negotiate lower speeds. Use the 1 Gbps ports for 1 Gbps networks.
Other ports
An SD-WAN 4100/5100 appliance has atleast two non-accelerated ports. Port 0/1 is typically used for management, Port 0/2 is present but typically not used. A Light Out Management (LOM) port is also provided. An RS-232 port can be used for management.