Network Experience Monitoring

Overview

Network Experience Monitoring (NEM) service (formerly called Netscope) enables service providers, enterprises, ISPs, and third party service providers to access detailed Radar measurement logs, and standard reports in the form of summarized actionable data. NEM offers several standard logs and reports that customers can use to measure the quality of their services.

This solution includes “raw” Radar Measurement delivery and access to the ITM Data API. NEM provides both the granular data (as either raw measurements or data aggregates) and data threshold alerts. These services help with discovery, isolate platform availability, and performance issues in platform peers and the underlying ISPs.

Radar “Raw” Measurements: Radar measurements provide per-event granular information that is batched daily. Radar measurements include public community and private measurement data collected by the tag. Data such as availability, response time, throughput for HTTP and HTTPS measurements are included. The following data fields are provided:

• Provider ID, resolver IP, obfuscated (/28) client IPs
• Obfuscated referrer header, user agent, end-user ASN
• Geo data for resolver and client fields

Radar Metrics that are available in the “Raw” Measurements are:

• Availability, Response Time, and Throughput (when measured)
• DNS Lookup Time (optional), TCP Connect Time (optional), and Secure Connect Time (optional)
• Latency (optional)
• Download Time (optional)

Radar Measurements are available to allow customers to do their own analysis of collected data. The data set includes information on provider performance and availability (errors) for a range of communication protocols.

Log file data is available for 7 days, from an AWS S3 or Google Cloud Storage bucket. Customers can retrieve log files of community and private data using standard bucket access methods.

Real-time Radar “Raw” Measurements (optional): Raw Radar measurements are delivered in real-time to an AWS S3 bucket. These logs are generally available within 5 minutes of collection. They provide as much granularity as the Radar Raw Measurements noted earlier.

Data API: The ITM Radar data API provides aggregates of the Radar public community and private measurement data. Data is updated continuously, and batched approximately every 60 seconds for retrieval by the API. The data API is provided to allow customers to integrate Radar data into their own reports and dashboards.

Log Sharing and Delivery

• Radar logs can be delivered real-time and daily.
• Reports run daily.
• Results are saved to AWS S3 (S3) or Google Cloud Storage (GCS).
• Logs and reports both have a 7-day retention period and are automatically deleted one week after creation.
• Reports are usually in TSV (tab-separated value) or JSON format depending on the type of report.

Customers are given login information to access the S3 and GCS buckets. A command line tool like s3cmd or the AWS CLI for S3 or the gsutil for GCS can be used to log in. The S3cmd config file recognizes the access keys received via the Portal UI and helps the user connect to the S3 bucket.

The AWS CLI needs to be installed in the customer’s computer to connect to S3 and access the logs. For GCS, the customer receives the access key file as a download via the Portal UI which can be used with the gsutil tool. For more information refer to the FAQ.

Customers receive email notifications as and when reports are available.

Platform Settings

You must configure your platform to support and produce the data required for Netscope NEM. Before you start, make sure that the following settings are enabled for your platform:

• For Resource Timing Details enable Include Timestamps in Advanced Radar Settings.

Navigation

From the main menu select Netscope NEM. The Network Experience Monitoring Configuration page opens.

Platforms

Select required Platforms to start the configuration process.

NOTE:

Logs and Reports can be configured and generated only if at least one Platform or Network is selected.

The summarized data that the customer receives include Radar measurements of selected Platforms (for all associated networks).

Selecting Platforms

For content service providers or enterprises, select platforms such as CDNs, clouds, data centers, or other end-points. Select platforms for which measurements are required.

Radar Logs

• Radar logs are available for Platforms.
• They include a subset of the fields available in the raw logs, with some data anonymized: client IP /28, Referer MD5 hashed.
• Every measurement taken for public platforms is provided, regardless of the page that generated the measurement.

NOTE:

NEM never exposes full client IPs. Instead, it exposes the /28. For example, an IP of 255.255.255.255 is shown in a report as 255.255.255.240/28.

Log Frequency

Radar Logs can be generated daily (every 24 hours) i.e.end of day, UTC time. Logs can also be generated in real-time (minute by minute).

File Format

Choose TSV or JSON to receive logs and reports in either of these formats.

Measurement Type

You can configure logs for the following measurement types: Availability, Response Time, and Throughput. In the report, 1: Availability, 0: HTTP Response Time, and 14: HTTP Throughput.

Resource Timing Details

You can choose to also include Resource Timing details by clicking the Yes or No buttons.Resource timing details include,

• DNS Lookup Time
• TCP Connection Time
• Secure Connection Time
• Download Time

For log descriptions refer to Radar Log Descriptions and Reports for Service Providers and Enterprises.

Navigation Timing Logs

Log Frequency

Navigation Timing Logs can be generated daily (every 24 hours) that is, end of day, UTC time. Logs can also be generated in real-time (minute by minute).

File Format

Choose TSV or JSON to receive Navigation Timing logs in either of these formats. For log descriptions refer to Navigation Timing Log Descriptions.

Openmix Logs

Log Frequency

Openmix logs are generated in real-time (that is, minute by minute). These logs provide real-time measurements taken for Openmix customers.

File Format

Choose TSV or JSON to receive Openmix and HTTP Openmix logs in either of these formats. JSON is however the recommended format.

For log descriptions refer to Openmix Log Descriptions.

Cloud Service Delivery

This option allows you to select the mode of delivery. You can choose to receive logs and reports in either the AWS S3 bucket or in the Google Cloud Storage (GCS) bucket. You can access the S3 and GCS buckets with the login information provided and use s3cmd or the AWS CLI for S3. and gsutil command line for GCS.

AWS S3

For logs and reports to be delivered to the AWS S3 bucket, select AWS S3.

Location

The Location represents the bucket in AWS S3 where the logs and reports are saved.

IAM Keys

If you select the Generate Keys button under AWS S3, the AWS IAM Keys (Access and Secret keys) are generated and displayed under IAM Keys. Be sure to record the keys because they aren’t stored anywhere for viewing later.

NOTE:

The pair of access and secret keys are the only copy of the private keys. The customer must store them securely. Regenerating the new keys invalidate the existing ones. The S3cmd config file recognizes the access keys (received via the Portal UI) and helps the customer connect to the S3 bucket. The AWS CLI needs to be installed in the customer’s machine to connect to the S3.

For info on how to use the access and secret keys with s3cmd to download reports from the S3 bucket, refer to the FAQ.

Google Cloud Storage

For logs and reports to be delivered to GCS, select Google Cloud Storage.

Location

The Location represents the bucket in Google Cloud Storage where logs and reports are saved.

IAM Keys

When you select the Generate Key File button, the Google Service Account Key File is downloaded to your machine.

NOTE:

This key file serves as the only copy of the private key. Take note of your service account’s email address and securely store the service account’s private key file. Regenerating a new key file invalidates the existing file.

This Key File can be used with the gsutil tool to download logs and reports from the GCS bucket. For details on how to use the Key File to download log files, refer to the FAQ.

Radar Log Descriptions and Reports for Service Providers and Enterprises

Radar Logs for Providers

• These logs provide Radar measurements for benchmark partners.
• They provide every measurement taken for public platforms, regardless of the page that generated the measurement.
• Radar logs include a subset of the fields available in the raw logs, with some data anonymized: client IP /28, Referer MD5 hashed.
• Here is a sample Platform Radar Log Share in TSV file format.

NOTE:

• NEM never exposes full client IPs. Instead, it exposes the /28. For example, an IP of 255.255.255.255 is shown in a report as 255.255.255.240/28.
• The client’s GEO information is extracted based on the client’s IPv4 which is more detailed.

Log Descriptions

The following are the columns headers and descriptions for the Radar Logs. The fields appear in the following order in the output files:

Navigation Timing Log Descriptions

Navigation Timing Data

Navigation Timing data provides insights into the various parts of the page load process for a webpage.

This data varies because of the end user’s location, network issues, changes made by the provider, and so on. Customers can use Navigation Timing data to optimize the end user’s experience in loading the monitored webpage.

Measurements can be taken for every Radar session (if enabled). Each session is attached to an ID number that helps track all measurements from a session. These measurements are shared with customers as Navigation Timing Logs via NEM.

The following is a sample of the Navigation Timing Data in TSV file format.

The following are the columns headers and descriptions for Navigation Timing logs. The fields appear in the following order in the output files:

Openmix and HTTP Openmix Logs

Openmix and HTTP Openmix logs allow customers to use real-time measurements to monitor the behavior of their Openmix apps. They can use this data to find areas of improvements or to verify the expected performance of their apps.

• These logs provide real-time measurements taken for Openmix customers.
• The recommended file format for these logs is JSON, but they’re available in TSV format as well.
• Here are samples of Openmix and HTTP Openmix log sharing data in TSV file format.

Openmix Log Descriptions

Openmix HTTP API Log Descriptions

Custom Reports for Third Party Organizations

Customers can work with NetScaler to get custom reports based on Radar data that NetScaler collects. NetScaler can generate reports to run on a schedule. The reports are available as data files, usually in TSV format.

FAQ

Radar

How frequently are files pushed to S3 and GCS?

The frequency of file deposits is once a minute for Radar and daily for reports.

Where are the reports stored?

S3 Legacy (Location 1):

s3://public-radar/[customer name]/

S3 (Location 2):

s3://cedexis-netscope/[customer id]/

GCS (Location 3):

gs://cedexis-netscope-[customer id]/

How to get S3 access credentials if you don’t have them already?

The portal provides an ‘Access’ and ‘Secret’ key. Use the keys with ‘s3cmd’, ‘awscli’ or other tools to access S3. For Google Storage, the Portal downloads a file with access credentials to use with the ‘gsutil’ tool.

How to use the access and secret keys with s3cmd to download logs and reports from the S3 bucket?

First you would need to download and install the s3cmd from https://s3tools.org/download, and refer to https://s3tools.org/usage for usage, options, and commands. Then run the following command:

s3cmd --access_key=[access key] --secret_key=[secret key] ls s3://cedexis-netscope/<customer id>/radar/
<!--NeedCopy-->

To download the files, run the following command:

s3cmd --access_key=[access_key] --secret_key=[secret_key] get s3://cedexis-netscope/<customer id>/radar/[the_filename_to_download] [the_name_of_the_local_file]
<!--NeedCopy-->

How to use the s3cmd config to list files in the S3 bucket

The first step is to Install s3cmd. You can install it from http://s3tools.org/download

To configure s3cmd, run the following command

s3cmd ls s3://cedexis-netscope/[customer id]/
<!--NeedCopy-->

If you’re already using s3cmd with another set of access and secret keys, then follow these steps:

If you already use s3cmd, then make a copy of the default config, at ~/.s3cfg. For example, make a copy and name it as ~/.s3cfg_netscope. Replace the access and secret key entries in ~/.s3cfg_netscope with the ones that we provide. Use the new configuration instead of the default one (your company’s) to access the S3 bucket with the following command:

s3cmd -c ~/.s3cfg_netscope ls s3://cedexis-netscope/[customer id]/
<!--NeedCopy-->

The main difference is you have to put in a -c and where the config file is with the NetScaler-provided access and secret keys.

If you want to switch between sets of keys, embed them in a file. Refer to the file with the -c option to specify which key pair you’re using.

NOTE: -c parameter indicates where the config file, that contains the access and secret keys, is located.

How to use the Key File with gsutil or gcloud to download log files

Once you download the google service account JSON Key File, you can use it to authenticate your google account credentials, view, or download your log files. For example, here’s one way to do that using the google gcloud and gsutil command line utilities:

Step 1: Activate the Key File

The authenticating commands gcloud auth activate- or gsutil config -e are required to authenticate the key file for running gcloud or gsutil commands.

For gcloud:

Run the following command using the downloaded Key File:

gcloud auth activate-service-account --key-file [downloaded config file]
<!--NeedCopy-->

Or

gcloud auth activate-service-account --key-file=[path and file name of key file]
<!--NeedCopy-->

For gsutil:

Run the following command using the downloaded config file:

gsutil config -e
<!--NeedCopy-->

Step 2: List the files in the GCS (Google Cloud Storage) Bucket

Once you’ve activated the service account key file as described in the earlier step, use the following command to list the files in the GCS bucket:

gsutil ls gs://cedexis-netscope-<customer id>
<!--NeedCopy-->

Step 3 (if necessary): Restore original credentials (or switch back and forth between accounts)

You can switch between the NetScaler ITM account and other Google Cloud credentials you’ve authenticated by doing the following.

First, run the following command to list all your accounts:

gcloud auth list
<!--NeedCopy-->

Then use the following command to switch to another account:

gcloud config set account [email of the account to switch to as shown in gcloud auth list]
<!--NeedCopy-->

You can switch back and forth between accounts using the same command, by replacing the email with the account email that you want to switch to.

What does the file name look like?

Legacy Daily:

The Radar daily log ShareFile names have this structure:

<prefix><date: YYYY-MM-DD>.<customer_id>.part<uniq_id>.kr.txt.gz

For example Cedexis_Daily-2017-11-07.21222.part-cc901e1dd55eal4e.kr.txt.gz (non-standard example)

Legacy Real-time:

The Radar real-time log ShareFile names have this structure:

<prefix><customer_id>-YYYY-MM-DDTHH:MM<uniq_id>.txt.gz

For example Cedexis_3-32291-2017-11-08T20:56-cc907e8fd71eaf4e.txt.gz

Netscope NEM Format:

The Netscope NEM format for daily and real-time log share files have this structure:

<freq><log_type><prefix><id_type><id><iso_dt><uniq_id>.<line_format>.gz

Where,

• freq: "daily" | "rt" | "hr"
• log_type: "radar" | "opx" | "hopx"
• prefix: log_share.prefix
• id_type: "customer" | "provider" | "asn"
• id: log_share.match_id
• iso_dt: iso 8601 Date_time "YYYYMMDDTHHMMSSZ"
• uniq_id: hash(UUID)
• line_format: "tsv" | "json"

For example rt-radar-TestRadar1-provider-20363-20171209183034Z-cc907e8fd71eaf4e.tsv.gz

What is the format of the output file?

For Radar, the output file format is TSV (tab-separated value), gzipped.

Openmix and Openmix HTTP API

How frequently are files pushed to S3?

The frequency of file deposits is once a minute for Openmix and HTTP Openmix.

What if you’re unable to see the option to configure Openmix and Openmix HTTP API real time log sharing?

Your Account Manager can enable the required role for you to configure and enable Openmix and Openmix HTTP API real time log sharing.

How do you turn on Openmix and an Openmix HTTP API real time log sharing and access files?

Once the role is enabled on your account, you see the Manage Logs icon. Click to open the Logs dialog where you can access Openmix Log Configuration settings. These settings are basically all you need to turn on Openmix and HTTP Openmix real time log-sharing and access files.

What is the back-end process?

Turning on Openmix log sharing enables Openmix HTTP API log sharing as well. The Openmix and Openmix HTTP API log-sharing services must start outputting logs for the customer within 10 minutes.

Where are the Openmix and HTTP Openmix reports stored?

S3 Legacy (Location 1):

s3://logshare/[zone ID]/[customer ID]/logs/openmix/json/[YYYY]/[MM]/[DD]/[HH]/.

S3 (Location 2):

s3://cedexis-netscope/[customer id]/

GCS (Location 3):

gs://cedexis-netscope-[customer id]/

What does the file name look like?

The file name structure for Openmix and HTTP Openmix typically looks like this:

Legacy Real-time:

[zone ID, 1][customerID]-openmix-json[YYYY][MM][DD][HH][mm][ss]Z-m1-w9-c0.gz

Netscope NEM Format:

The Netscope NEM format for daily and real-time log share files have this structure:

<freq><log_type><prefix><id_type><id><iso_dt><uniq_id>.<line_format>.gz

Where,

• freq: "daily" | "rt" | "hr"
• log_type: "radar" | "opx" | "hopx"
• prefix: log_share.prefix
• id_type: "customer" | "provider" | "asn"
• idv: log_share.match_id
• iso_dt: iso 8601 Date_time "YYYYMMDDTHHMMSSZ"
• uniq_id: hash(UUID)
• line_format: "tsv" | "json"

For example hr-opx-TestOpenmix1-provider-20363-20171209183034Z-cc907e8fd71eaf4e.tsv.gz

What is the output file format?

The file format for Openmix and an Openmix HTTP API is JSON (gzipped).