AppFlow and IPFIX
AppFlow and IPFIX are flow export standards used to identify and collect application and transaction data in the network infrastructure. This data gives better visibility into application traffic utilization and performance.
The collected data, called flow records are transmitted to one or more IPv4 or IPv6 collectors. The collectors aggregate the flow records and generate real-time or historical reports.
AppFlow
AppFlow exports flow level data for HDX / ICA connections only. You can enable either the TCP only for HDX dataset template or the HDX dataset template. The TCP only for HDX dataset provides multi-hop data. The HDX dataset provides HDX insight data.
AppFlow Collectors like Splunk and Citrix ADM have dashboards to interpret and present these templates.
IPFIX
IPFIX is a collector export protocol used for exporting flow level data for all connections. For any connection, you can view information such as packet count, byte count, type of service, flow direction, routing domain, application name and so on. IPFIX flows are transmitted through the management interface. Most collectors can receive IPFIX flow records, but may need to build a custom dashboard to interpret IPFIX template.
The IPFIX template defines the order in which the data stream is to be interpreted. The collector receives a template record, followed by the data records. Citrix SD-WAN uses templates 611 and 613 to export IPv4 IPFIX flow data, 615 and 616 to export IPv6 IPFIX flow data along with Options template 612.
Application Flow Info (IPFIX) exports data sets as per templates 611 for IPv4 flows, 615 for IPv6 flows and 612 options Template with Application info.
Basic Properties (IPFIX) exports data sets as per templates 613 for IPv4 flows and 616 for IPv6 flows.
The following tables provide the detailed list of flow data associated with each IPFIX template.
Application Flow Info (IPFIX) - V10 templates
Template ID - 611
| Info Element (IE) | IE name & ID | Type and len | Description |
|---|---|---|---|
| Observation point ID | observationPointId, 138 | Unsigned32, 4 | |
| Export process ID | exportingProcessId, 144 | Unsigned32, 4 | |
| Flow ID | flowId, 148 | Unsigned64, 8 | |
| Ipv4 SRC IP | sourceIPv4Address, 8 | Ipv4address, 4 | |
| Ipv4 DST IP | destinationIpv4Addres, 12 | Ipv4address, 4 | |
| Ipversion | ipVersion, 60 | Unsigned8, 1 | Set to 4. |
| IP protocol number | protocoldentifier,4 | Unsigned8, 1 | |
| Padding | N/A | Unsigned16, 2 | |
| SRC Port | sourceTransportPort, 7 | Unsigned16, 2 | |
| DST Port | destinationTransportPort,11 | Unsigned16, 2 | |
| Pkt Count | packetDeltaCount, 2 | Unsigned64, 8 | |
| Byte Count | octetDeltaCount, 1 | Unsigned64, 8 | |
| Time for first pkt in microseconds | flowStartMicroseconds, 154 | dateTimeMicroseconds, 8 | |
| Time for lastpkt in microseconds | flowEndMicroseconds, 155 | dateTimeMicroseconds, 8 | |
| IP ToS | ipClassOfService, 5 | Unsigned8, 1 | |
| Flow Flags | tcpControlBits, 6 | Unsigned8, 2 | Currently set to 0. |
| Flow Direction | flowDirection, 61 | Unsigned8, 1 | 0x00: ingress flow0x01: egress flowWAN-WAN and LAN-LAN flows are a possibility in SDWAN |
| Input Interface | ingressInterface, 10 | Unsigned32, 4 | Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations. |
| Output Interface | egressInterface, 14 | Unsigned32, 4 | Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations. |
| Input Vlan ID | vlanId, 58 | Unsigned16, 2 | |
| Output Vlan ID | postVlanId, 59 | Unsigned16, 2 | |
| VRF ID | ingressVRFID, 234 | Unsigned32, 4 | |
| Flow Key Indicator | flowKeyIndicator, 173 | Unsigned64, 8 | Set to 0x1E037F. |
| Application ID | applicationId, 95 | octetArray, variable | The Application ID is same as the ID of the applications classified by the DPI engine. The application IDs remain constant. The application IDs for Custom domain name based applications change with every configuration update. |
Template ID – 615 (IPv6 flows)
| Info Element (IE) | IE name & ID | Type and len | Comment | |
|---|---|---|---|---|
| Observation point ID | observationPointId, 138 | Unsigned32, 4 | ||
| Export process ID | exportingProcessId, 144 | Unsigned32, 4 | ||
| Flow ID | flowId, 148 | Unsigned64, 8 | ||
| Ipv6 SRC IP | sourceIPv6Address, 27 | Ipv6address, 16 | ||
| Ipv6 DST IP | destinationIpv6Addres, 28 | Ipv6address, 16 | ||
| Ipversion | ipVersion, 60 | Unsigned8, 1 | Set to 6 | |
| IP protocol number | protocoldentifier, 4 | Unsigned8, 1 | ||
| Padding | N/A | Unsigned16, 2 | ||
| SRC Port | sourceTransportPort, 7 | Unsigned16, 2 | ||
| DST Port | destinationTransportPort, 11 | Unsigned16, 2 | ||
| Pkt Count | packetDeltaCount, 2 | Unsigned64, 8 | ||
| Byte Count | octetDeltaCount, 1 | Unsigned64, 8 | ||
| Time for first pkt in microseconds | flowStartMicroseconds, 154 | dateTimeMicroseconds, 8 | ||
| Time for lastpkt in microseconds | flowEndMicroseconds, 155 | dateTimeMicroseconds, 8 | ||
| IP ToS | ipClassOfService, 5 | Unsigned8, 1 | ||
| Flow Flags | tcpControlBits, 6 | Unsigned8, 2 | Currently set to 0. | |
| Flow Direction | flowDirection, 61 | Unsigned8, 1 | 0x00: ingress flow0x01: egress flowWAN-WAN and LAN-LAN flows are a possibility in SDWAN | |
| Input Interface | ingressInterface, 10 | Unsigned32, 4 | Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations. | |
| Output Interface | egressInterface, 14 | Unsigned32, 4 | Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations. | |
| Input Vlan ID | vlanId, 58 | Unsigned16, 2 | ||
| Output Vlan ID | postVlanId, 59 | Unsigned16, 2 | ||
| VRF ID | ingressVRFID, 234 | Unsigned32, 4 | ||
| Flow Key Indicator | flowKeyIndicator, 173 | Unsigned64, 8 | Set to 0x1E037F. | |
| Application ID | applicationId, 95 | octetArray, variable | The Application ID is same as the ID of the applications classified by the DPI engine. The application IDs remain constant. The application IDs for Custom domain name based applications change with every configuration update. | |
Template 612 (Options Template)
| Info Element (IE) | IE name & ID | Type | Comment |
|---|---|---|---|
| Application ID | applicationId, 95 | octetArray | The Application ID is same as the ID of the applications classified by the DPI engine. The application IDs remain constant. The application IDs for Custom domain name based applications change with every configuration update. |
| Application Name | applicationName, 96 | string | Specifies the name of the Citrix SDWAN specific proprietary application. |
| Application Description | applicationDescription, 94 | string | Specifies the description of the application. |
Basic Properties (IPFIX) – V9 compliant template - Template 613 (IPv4 flows)
| Info Element (IE) | IE name & ID | Type and len | Comment |
|---|---|---|---|
| Ipv4 SRC IP | sourceIPv4Address, 8 | Ipv4address, 4 | |
| Ipv4 DST IP | destinationIpv4Addres, 12 | Ipv4address, 4 | |
| Ipversion | ipVersion, 60 | Unsigned8, 1 | |
| IP protocol number | protocoldentifier, 4 | Unsigned8, 1 | |
| IP ToS | ipClassOfService, 5 | Unsigned8, 1 | |
| Flow Direction | flowDirection, 61 | Unsigned8, 1 | 0x00: ingress flow0x01: egress flowWAN-WAN and LAN-LAN flows are a possibility in SDWAN |
| SRC Port | sourceTransportPort, 7 | Unsigned16, 2 | |
| DST Port | destinationTransportPort, 11 | Unsigned16, 2 | |
| Pkt Count | packetDeltaCount, 2 | Unsigned64, 8 | |
| Byte Count | octetDeltaCount, 1 | Unsigned64, 8 | |
| Input Interface | ingressInterface, 10 | Unsigned32, 4 | Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations. |
| Output Interface | egressInterface, 14 | Unsigned32, 4 | Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations. |
| Input Vlan ID | vlanId, 58 | Unsigned16, 2 | |
| Output Vlan ID | postVlanId, 59 | Unsigned16, 2 |
Template ID – 616 (IPv6 flows)
| Info Element (IE) | IE name & ID | Type and len | Comment | |
|---|---|---|---|---|
| Ipv6 SRC IP | sourceIPv6Address, 27 | Ipv6address, 16 | ||
| Ipv6 DST IP | destinationIpv6Addres, 28 | Ipv6address, 16 | ||
| Ipversion | ipVersion, 60 | Unsigned8, 1 | Set to 6 | |
| IP protocol number | protocoldentifier,4 | Unsigned8, 1 | ||
| IP ToS | ipClassOfService, 5 | Unsigned8, 1 | ||
| Flow Direction | flowDirection, 61 | Unsigned8, 1 | 0x00: ingress flow0x01: egress flowWAN-WAN and LAN-LAN flows are a possibility in SDWAN | |
| SRC Port | sourceTransportPort, 7 | Unsigned16, 2 | ||
| DST Port | destinationTransportPort, 11 | Unsigned16, 2 | ||
| Pkt Count | packetDeltaCount, 2 | Unsigned64, 8 | ||
| Byte Count | octetDeltaCount, 1 | Unsigned64, 8 | ||
| Input Interface | ingressInterface, 10 | Unsigned32, 4 | Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations. | |
| Output Interface | egressInterface, 14 | Unsigned32, 4 | Citrix SD-WAN load balances data flows through multiple member paths, hence a single data flow can have multiple input/output interface combinations. | |
| Input Vlan ID | vlanId, 58 | Unsigned16, 2 | ||
| Output Vlan ID | postVlanId, 59 | Unsigned16, 2 |
Limitations
- AppFlow does not support IPv6 collector and flow records.
- The export interval for Net Flow is increased from 15 seconds to 60 seconds.
- AppFlow/IPFIX flows are transmitted over UDP, on connection loss not all data is retransmitted. If the export interval is set to X minutes, the appliance stores X minutes of data only. Which is retransmitted after X minutes of connection loss.
- In Citrix SD-WAN, release 10 version 2 the AppFlow settings are made local to every appliance, while in the previous releases it was a global setting. If the SD-WAN software release is downgraded to any of the previous releases and if AppFlow is configured on any one of the appliances, it will be applied globally to all alliances.
Configuring AppFlow/IPFIX
You can configure AppFlow / IPFIX only through Citrix SD-WAN Orchestrator service. For more informtion, see AppFlow and IPFIX.
Log files
For troubleshooting issues related to AppFlow / IPFIX export protocols, you can view and download the SDWAN_export.log files. Navigate to Configuration > Logging / Monitoring and select the SDWAN_export.log files.
