AppFlow and IPFIX
AppFlow and IPFIX are flow export standards used to identify and collect application and transaction data in the network infrastructure. This data gives better visibility into application traffic utilization and performance.
The collected data, called flow records are transmitted to one or more IPv4 or IPv6 collectors. The collectors aggregate the flow records and generate real-time or historical reports.
AppFlow
AppFlow exports flow level data for HDX / ICA connections only. You can enable either the TCP only for HDX dataset template or the HDX dataset template. The TCP only for HDX dataset provides multi-hop data. The HDX dataset provides HDX insight data.
Note
HDX template is available for Citrix SD-WAN PE edition and Two-box appliances only. It should be enabled on the Data Center appliance.
AppFlow Collectors like Splunk and Citrix ADM have dashboards to interpret and present these templates.
IPFIX
IPFIX is a collector export protocol used for exporting flow level data for all connections. For any connection, you can view information such as packet count, byte count, type of service, flow direction, routing domain, application name and so on. IPFIX flows are transmitted through the management interface. Most collectors can receive IPFIX flow records, but may need to build a custom dashboard to interpret IPFIX template.
The IPFIX template defines the order in which the data stream is to be interpreted. The collector receives a template record, followed by the data records. Citrix SD-WAN uses templates 611 and 613 to export IPv4 IPFIX flow data, 615 and 616 to export IPv6 IPFIX flow data along with Options template 612.
Application Flow Info (IPFIX) exports data sets as per templates 611 for IPv4 flows, 615 for IPv6 flows and 612 options Template with Application info.
Basic Properties (IPFIX) exports data sets as per templates 613 for IPv4 flows and 616 for IPv6 flows.
The following tables provide the detailed list of flow data associated with each IPFIX template.
Application Flow Info (IPFIX) - V10 templates
Template ID - 611
| Info Element (IE) | IE name & ID | Type and len | Description |
|---|---|---|---|
| Observation point ID | observationPointId, 138 | Unsigned32, 4 | |
| Export process ID | exportingProcessId, 144 | Unsigned32, 4 | |
| Flow ID | flowId, 148 | Unsigned64, 8 | |
| Ipv4 SRC IP | sourceIPv4Address, 8 | Ipv4address, 4 | |
| Ipv4 DST IP | destinationIpv4Addres, 12 | Ipv4address, 4 | |
| Ipversion | ipVersion, 60 | Unsigned8, 1 | Set to 4. |
| IP protocol number | protocoldentifier,4 | Unsigned8, 1 | |
| Padding | N/A | Unsigned16, 2 | |
| SRC Port | sourceTransportPort, 7 | Unsigned16, 2 | |
| DST Port | destinationTransportPort,11 | Unsigned16, 2 | |
| Pkt Count | packetDeltaCount, 2 | Unsigned64, 8 | |
| Byte Count | octetDeltaCount, 1 | Unsigned64, 8 | |
| Time for first pkt in microseconds | flowStartMicroseconds, 154 | dateTimeMicroseconds, 8 | |
| Time for lastpkt in microseconds | flowEndMicroseconds, 155 | dateTimeMicroseconds, 8 | |
| IP ToS | ipClassOfService, 5 | Unsigned8, 1 | |
| Flow Flags | tcpControlBits, 6 | Unsigned8, 2 | Currently set to 0. |
| Flow Direction | flowDirection, 61 | Unsigned8, 1 | 0x00: ingress flow0x01: egress flowWAN-WAN and LAN-LAN flows are a possibility in SDWAN |
| Input Interface | ingressInterface, 10 | Unsigned32, 4 | Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations. |
| Output Interface | egressInterface, 14 | Unsigned32, 4 | Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations. |
| Input Vlan ID | vlanId, 58 | Unsigned16, 2 | |
| Output Vlan ID | postVlanId, 59 | Unsigned16, 2 | |
| VRF ID | ingressVRFID, 234 | Unsigned32, 4 | |
| Flow Key Indicator | flowKeyIndicator, 173 | Unsigned64, 8 | Set to 0x1E037F. |
| Application ID | applicationId, 95 | octetArray, variable | The Application ID is same as the ID of the applications classified by the DPI engine. The application IDs remain constant. The application IDs for Custom domain name based applications change with every configuration update. |
Template ID – 615 (IPv6 flows)
| Info Element (IE) | IE name & ID | Type and len | Comment | |
|---|---|---|---|---|
| Observation point ID | observationPointId, 138 | Unsigned32, 4 | ||
| Export process ID | exportingProcessId, 144 | Unsigned32, 4 | ||
| Flow ID | flowId, 148 | Unsigned64, 8 | ||
| Ipv6 SRC IP | sourceIPv6Address, 27 | Ipv6address, 16 | ||
| Ipv6 DST IP | destinationIpv6Addres, 28 | Ipv6address, 16 | ||
| Ipversion | ipVersion, 60 | Unsigned8, 1 | Set to 6 | |
| IP protocol number | protocoldentifier, 4 | Unsigned8, 1 | ||
| Padding | N/A | Unsigned16, 2 | ||
| SRC Port | sourceTransportPort, 7 | Unsigned16, 2 | ||
| DST Port | destinationTransportPort, 11 | Unsigned16, 2 | ||
| Pkt Count | packetDeltaCount, 2 | Unsigned64, 8 | ||
| Byte Count | octetDeltaCount, 1 | Unsigned64, 8 | ||
| Time for first pkt in microseconds | flowStartMicroseconds, 154 | dateTimeMicroseconds, 8 | ||
| Time for lastpkt in microseconds | flowEndMicroseconds, 155 | dateTimeMicroseconds, 8 | ||
| IP ToS | ipClassOfService, 5 | Unsigned8, 1 | ||
| Flow Flags | tcpControlBits, 6 | Unsigned8, 2 | Currently set to 0. | |
| Flow Direction | flowDirection, 61 | Unsigned8, 1 | 0x00: ingress flow0x01: egress flowWAN-WAN and LAN-LAN flows are a possibility in SDWAN | |
| Input Interface | ingressInterface, 10 | Unsigned32, 4 | Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations. | |
| Output Interface | egressInterface, 14 | Unsigned32, 4 | Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations. | |
| Input Vlan ID | vlanId, 58 | Unsigned16, 2 | ||
| Output Vlan ID | postVlanId, 59 | Unsigned16, 2 | ||
| VRF ID | ingressVRFID, 234 | Unsigned32, 4 | ||
| Flow Key Indicator | flowKeyIndicator, 173 | Unsigned64, 8 | Set to 0x1E037F. | |
| Application ID | applicationId, 95 | octetArray, variable | The Application ID is same as the ID of the applications classified by the DPI engine. The application IDs remain constant. The application IDs for Custom domain name based applications change with every configuration update. | |
Template 612 (Options Template)
| Info Element (IE) | IE name & ID | Type | Comment |
|---|---|---|---|
| Application ID | applicationId, 95 | octetArray | The Application ID is same as the ID of the applications classified by the DPI engine. The application IDs remain constant. The application IDs for Custom domain name based applications change with every configuration update. |
| Application Name | applicationName, 96 | string | Specifies the name of the Citrix SDWAN specific proprietary application. |
| Application Description | applicationDescription, 94 | string | Specifies the description of the application. |
Basic Properties (IPFIX) – V9 compliant template - Template 613 (IPv4 flows)
| Info Element (IE) | IE name & ID | Type and len | Comment |
|---|---|---|---|
| Ipv4 SRC IP | sourceIPv4Address, 8 | Ipv4address, 4 | |
| Ipv4 DST IP | destinationIpv4Addres, 12 | Ipv4address, 4 | |
| Ipversion | ipVersion, 60 | Unsigned8, 1 | |
| IP protocol number | protocoldentifier, 4 | Unsigned8, 1 | |
| IP ToS | ipClassOfService, 5 | Unsigned8, 1 | |
| Flow Direction | flowDirection, 61 | Unsigned8, 1 | 0x00: ingress flow0x01: egress flowWAN-WAN and LAN-LAN flows are a possibility in SDWAN |
| SRC Port | sourceTransportPort, 7 | Unsigned16, 2 | |
| DST Port | destinationTransportPort, 11 | Unsigned16, 2 | |
| Pkt Count | packetDeltaCount, 2 | Unsigned64, 8 | |
| Byte Count | octetDeltaCount, 1 | Unsigned64, 8 | |
| Input Interface | ingressInterface, 10 | Unsigned32, 4 | Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations. |
| Output Interface | egressInterface, 14 | Unsigned32, 4 | Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations. |
| Input Vlan ID | vlanId, 58 | Unsigned16, 2 | |
| Output Vlan ID | postVlanId, 59 | Unsigned16, 2 |
Template ID – 616 (IPv6 flows)
| Info Element (IE) | IE name & ID | Type and len | Comment | |
|---|---|---|---|---|
| Ipv6 SRC IP | sourceIPv6Address, 27 | Ipv6address, 16 | ||
| Ipv6 DST IP | destinationIpv6Addres, 28 | Ipv6address, 16 | ||
| Ipversion | ipVersion, 60 | Unsigned8, 1 | Set to 6 | |
| IP protocol number | protocoldentifier,4 | Unsigned8, 1 | ||
| IP ToS | ipClassOfService, 5 | Unsigned8, 1 | ||
| Flow Direction | flowDirection, 61 | Unsigned8, 1 | 0x00: ingress flow0x01: egress flowWAN-WAN and LAN-LAN flows are a possibility in SDWAN | |
| SRC Port | sourceTransportPort, 7 | Unsigned16, 2 | ||
| DST Port | destinationTransportPort, 11 | Unsigned16, 2 | ||
| Pkt Count | packetDeltaCount, 2 | Unsigned64, 8 | ||
| Byte Count | octetDeltaCount, 1 | Unsigned64, 8 | ||
| Input Interface | ingressInterface, 10 | Unsigned32, 4 | Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations. | |
| Output Interface | egressInterface, 14 | Unsigned32, 4 | Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations. | |
| Input Vlan ID | vlanId, 58 | Unsigned16, 2 | ||
| Output Vlan ID | postVlanId, 59 | Unsigned16, 2 |
Limitations
- AppFlow does not support IPv6 collector and flow records.
- The export interval for Net Flow is increased from 15 seconds to 60 seconds.
- AppFlow/IPFIX flows are transmitted over UDP, on connection loss not all data is retransmitted. If the export interval is set to X minutes, the appliance stores X minutes of data only. Which is retransmitted after X minutes of connection loss.
- In Citrix SD-WAN, release 10 version 2 the AppFlow settings are made local to every appliance, while in the previous releases it was a global setting. If the SD-WAN software release is downgraded to any of the previous releases and if AppFlow is configured on any one of the appliances, it will be applied globally to all alliances.
Configuring AppFlow/IPFIX
To configure AppFlow Host Settings, navigate to Configuration > Appliance Settings > AppFlow Host Settings and click Enable. Specify the data update interval, in minutes, at which the AppFlow reports are exported to the AppFlow / IPFIX collector.
Choose one of the following AppFlow dataset templates:
- TCP only for HDX: Collects and sends multi-hop data of ICA connections to the AppFlow collector.
- HDX: Collects and sends HDX insight data of ICA connections to the AppFlow collector.
You can configure up to four AppFlow / IPFIX collectors. For each collector specify the following parameters:
- IP Address: The IP address of the external AppFlow / IPFIX collector system.
- Port: The port number on which the external AppFlow / IPFIX collector system listens. The default value is 4739. You can change the port number depending on the collector used.
- AppFlow: Sends flow records, as per IPFIX template 613, to IPFIX collectors.
- Application Flow Info: Sends flow records, as per IPFIX templates 611 and 612, to IPFIX collectors.
- Citrix ADM: Use Citrix ADM as the AppFlow collector. Provide the user name and password to seamlessly log in into Citrix ADM and store flow data.
Note
Citrix ADM currently does not support IPFIX collection.
Log files
For troubleshooting issues related to AppFlow / IPFIX export protocols, you can view and download the SDWAN_export.log files from Site logs