Product Documentation
ADC
14.1 - Current Release 13.1 13.0 12.1
View PDF

IP Reputation

October 30, 2023
Contributed by: 
C S

IP reputation is a tool that identifies IP addresses that send unwanted requests. Using the IP reputation list you can reject requests that are coming from an IP address with a bad reputation. Optimize Web Application Firewall performance by filtering requests that you do not want to process. Reset, drop a request, or even configure a responder policy to take a specific responder action.

The following are some attacks that you can prevent by using IP Reputation:

  • Virus Infected personal computers. (home PCs) are the single biggest source of Spam on the internet. IP Reputation can identify the IP address that is sending unwanted requests. IP reputation can be especially useful for blocking large scale DDoS, DoS, or anomalous SYN flood attacks from known infected sources.
  • Centrally managed and automated botnet. Attackers have gained popularity for stealing passwords, because it doesn’t take long when hundreds of computers work together to crack your password. It is easy to launch botnet attacks to figure out passwords that use commonly used dictionary words.
  • Compromised web-server. Attacks are not as common because awareness and server security have increased, so hackers and spammers look for easier targets. There are still web servers and online forms that hackers can compromise and use to send spam (such as viruses and porn). Such activity is easier to detect and quickly shut down, or block with a reputation list such as SpamRats.
  • Windows Exploits. (such as Active IPs offering or distributing malware, shell code, rootkits, worms, or viruses).
  • Known spammers and hackers.
  • Mass e-mail marketing campaigns.
  • Phishing Proxies (IP addresses hosting phishing sites, and other fraud such as ad click fraud or gaming fraud).
  • Anonymous proxies (IPs providing proxy and anonymization services including The Onion Router aka TOR).

A NetScaler appliance uses Webroot as the service provider for a dynamically generated malicious IP database and the metadata for those IP addresses. Metadata might include geolocation details, threat category, threat count, and so on. The Webroot threat Intelligence engine receives real-time data from millions of sensors. It automatically and continuously captures, scans, analyses and scores the data, using advanced machine learning and behavioral analysis. Intelligence about a threat is continually updated.

The NetScaler appliance validates an incoming request for its bad reputation using the Webroot’s uses IP reputation database. The database has a huge collection of IP address classified based IP threat categories. Following are the IP threat categories and its description.

  • Spam Sources. Spam Sources includes Tunneling Spam messages through proxy, anomalous SMTP activities, Forum Spam activities.
  • Windows Exploits. The Windows Exploits category includes active IP Address offering or distributing malware, shell code, rootkits, worms, or viruses
  • Web Attacks. The Web Attacks category includes cross site scripting, iFrame injection, SQL injection, cross domain injection, or domain password brute force attack
  • Botnets. The Botnet category includes Botnet C&C channels, and infected zombie machine controlled by Bot master
  • Scanners. The Scanners category includes all reconnaissance such as probes, host scan, domain scan and password brute force attack
  • Denial of Service. The Denial of Services category includes DOS, DDOS, anomalous sync flood, anomalous traffic detection
  • Reputation. Deny access from IP addresses currently known to be infected with malware. This category also includes IPs with average low Webroot Reputation Index score. Enabling this category prevents access from sources identified to contact malware distribution points.
  • Phishing. The Phishing category includes IP addresses hosting phishing sites, other kinds of fraud activities such as Ad Click Fraud or Gaming fraud
  • Proxy. The Proxy category includes IP addresses providing proxy and def services.
  • Mobile Threats. The Mobile Threats category includes IP addresses of malicious and unwanted mobile applications. This category leverages data from the Webroot mobile threat research team.
  • Tor Proxy. The Tor Proxy category includes IP addresses acting as exit nodes for the Tor Network. Exit nodes are the last point along the proxy chain and make a direct connection to the originator’s intended destination.

When a threat is detected anywhere in the network, the IP address is flagged as malicious and all appliances connected to the network are immediately protected. The dynamic changes in the IP addresses are processed with high speed and accuracy by using advanced machine learning.

As stated in the data sheet from Webroot, the Webroot’s sensor network identifies many key IP threat types, including spam sources, Windows exploits, botnets, scanners, and others. (See the flow diagram on the data sheet.)

The NetScaler appliance uses an iprep client process to get the database from Webroot. The iprep client uses the HTTP GET method to get the absolute IP list from Webroot for the first time. Later, it checks delta changes once every 5 minutes.

Important:

  • Make sure that the NetScaler appliance has Internet access and DNS is configured before you use the IP Reputation feature.

  • To access the Webroot database, the NetScaler appliance must be able to connect to api.bcti.brightcloud.com on port 443. Each node in the HA or cluster deployment gets the database from Webroot and must be able to access this Fully Qualified Domain Name (FQDN).

  • Webroot hosts its reputation database in AWS currently. Therefore, NetScaler must be able to resolve AWS domains for downloading the reputation db. Also, the firewall must be open for AWS domains.

Note:

Each packet engine requires at least 4 GB to function properly when the IP Reputation feature is enabled.

Advanced policy Expressions. Configure the IP Reputation feature by using advanced policy expressions (Advanced policy expressions) in the policies bound to supported modules, such as Web Application Firewall and responder. The following are two examples showing expressions that can be used to detect whether the client IP address is malicious.

  1. CLIENT.IP.SRC.IPREP_IS_MALICIOUS: This expression evaluates to TRUE if the client is included in the malicious IP list.
  2. CLIENT.IP.SRC.IPREP_THREAT_CATEGORY (CATEGORY): This expression evaluates to TRUE if the client IP is malicious IP and is in the specified threat category.
  3. CLIENT.IPV6.SRC.IPREP_IS_MALICIOUS and CLIENT.IPV6.SRC.IPREP_THREAT_CATEGORY: This expression evaluates to TRUE if the client IP is of type IPv6 and it is a malicious IP address in a specified threat category.

Following are the possible values for the threat category:

SPAM_SOURCES, WINDOWS_EXPLOITS, WEB_ATTACKS, BOTNETS, SCANNERS, DOS, REPUTATION, PHISHING, PROXY, NETWORK, CLOUD_PROVIDERS, MOBILE_THREATS, TOR_PROXY.

Note:

The IP reputation feature checks both source and destination IP addresses. It detects malicious IPs in the header. If the PI Expression in a policy can identify the IP address, the IP reputation check determines whether it is malicious.

IPRep log message. The /var/log/iprep.log file contains useful messages that capture information about communication with the Webroot database. The information can be about the credentials used during Webroot communication, failure to connect with Webroot, information included in an update (such as the number of IP addresses in the database).

Creating a blocklist or allowlist of IPs using a policy data set. You can maintain an allow list to allow access to specific IP addresses that are blocklisted in the Webroot database. You can also create a customized block list of IP addresses to supplement the Webroot reputation check. These lists can be created by using a policy data set. A data set is a specialized form of pattern set that is ideally suited for IPv4 or IPv6 address matching. To use data sets, first create the data set and bind IPv4 or IPv6 addresses to it. When configuring a policy for comparing a string in a packet, use an appropriate operator and pass the name of the pattern set or data set as an argument.

To create an allow list of addresses to treat as exceptions during IP reputation evaluation:

  • Configure the policy so that the PI expression evaluates to False even if an address in the allow list is listed as malicious by Webroot (or any service provider).

Enabling or disabling IP reputation. IP reputation is a part of the general reputation feature, which is license based. When you enable or disable the reputation feature, it enables or disables IP Reputation.

General procedure. Deploying IP reputation involves the following tasks

  • Verify that the license installed on the NetScaler appliance has IP reputation support. Premium and standalone application firewall licenses support the IP reputation feature.
  • Enable the IP reputation and application firewall features.
  • Add an application firewall profile.
  • Add an application firewall policy using the PI expressions to identify the malicious IP addresses in the IP Reputation database.
  • Bind the application firewall policy to an appropriate bind point.
  • Verify that any request received from a malicious address gets logged in the ns.log file to show that the request was processed as specified in the profile.

Configure the IP reputation feature using the CLI

At the command prompt, type:

  • enable feature reputation
  • disable feature reputation

The following examples show how you can add an application firewall policy using the PI expression to identify malicious addresses. You can use the built-in profiles, or add a profile, or configure an existing profile to invoke the desired action when a request matches a policy match.

Examples 3 and 4 show how to create a policy dataset to generate a block list or an allow list of IP addresses.

Example 1:

The following command creates a policy that identifies malicious IP addresses and block the request if a match is triggered:

add appfw policy pol1 CLIENT.IP.SRC.IPREP_IS_MALICIOUS APPFW_BLOCK add appfw policy pol1 CLIENT.IPv6.SRC.IPREP_IS_MALICIOUS APPFW_BLOCK add appfw policy pol1 "HTTP.REQ.HEADER(\"X-Forwarded-For\") .TYPECAST_IPv6_ADDRESS_AT.IPREP_IS_MALICIOUS" APPFW_RESET

Example 2:

The following command creates a policy that uses the reputation service to check the client IP address in the X-Forwarded-For header and reset the connection if a match is triggered.

> add appfw policy pol1 "HTTP.REQ.HEADER(\"X-Forwarded-For\").TYPECAST_IP_ADDRESS_AT.IPREP_IS_MALICIOUS" APPFW_RESET**

Example 3:

The following example shows how to add a list to add exceptions that allow specified IP addresses:

> add policy dataset Allow_list1 ipv4

> bind policy dataset Allow_list1 10.217.25.17 -index 1

> bind policy dataset Allow_list1 10.217.25.18 -index 2

The following example shows how to add a list to add exceptions that allow specified IPv6 addresses:

add policy dataset Allow_list_ipv6 ipv6
bind policy dataset Allow_list_ipv6 fe80::98c7:d8ff:fe3a:b562 -index 1
bind policy dataset Allow_list_ipv6 fe80::98c7:d8ff:fe3a:b563  -index 2

<!--NeedCopy-->

Example 4:

The following example shows how to add the customized list to flag specified IP addresses as malicious:

> add policy dataset Block_list1 ipv4

> bind policy dataset Block_list1 10.217.31.48 -index 1

> bind policy dataset Block_list1 10.217.25.19 -index 2

The following example shows how to add the customized list to flag specified IPv6 addresses as malicious.

add policy dataset Block_list_ipv6 ipv6
bind policy dataset Block_list_ipv6 fe80::98c7:d8ff:ff3b:b562 -index 1
bind policy dataset Block_list_ipv6 fe80::ffc7:d8ff:fe3a:b562 -index 2
<!--NeedCopy-->

Example 5:

The following example shows a policy expression to block the client IP in the following conditions:

  • It matches an IP address configured in the customized Block_list1 (example 4)
  • It matches an IP address listed in the Webroot database unless relaxed by inclusion in the Allow_list1 (example 3).
> add appfw policy "Ip_Rep_Policy" "((CLIENT.IP.SRC.IPREP_IS_MALICIOUS || CLIENT.IP.SRC.TYPECAST_TEXT_T.CONTAINS_ANY(\"Block_list1\")) && ! (CLIENT.IP.SRC.TYPECAST_TEXT_T.CONTAINS_ANY(\"Allow_list1\")))" APPFW_BLOCK
<!--NeedCopy-->

The following example shows a policy expression to block the client IPv6 in the following conditions:

  1. It matches an IPv6 address configured in the customized Block_list_ipv6 (example 4)
  2. It matches an Ipv6 address listed in the Webroot database unless relaxed by inclusion in the Allow_list_ipv6 (example 3).
add appfw policy "Ip_Rep_v6_Policy" "((CLIENT.IPV6.SRC.IPREP_IS_MALICIOUS || CLIENT.IPV6.SRC.TYPECAST_TEXT_T.CONTAINS_ANY(\"Block_list_ipv6\")) && ! (CLIENT.IPV6.SRC.TYPECAST_TEXT_T.CONTAINS_ANY(\"Allow_list_ipv6\")))" APPFW_BLOCK
<!--NeedCopy-->

Using Proxy server:

If the NetScaler appliance does not have direct access to the internet and is connected to a proxy, configure the IP Reputation client to send requests to the proxy.

Configure a proxy user name and password on the proxy server for an additional layer of security for your appliance.

At the command prompt, type:

set reputation settings –proxyServer <proxy server ip> -proxyPort <proxy server port> -proxyUsername <username> -proxyPassword <password>

Example:

> set reputation settings proxyServer 10.102.30.112 proxyPort 3128 -proxyUsername defaultusername -proxyPassword defaultpassword

> set reputation settings –proxyServer testproxy.citrite.net –proxyPort 3128 -proxyUsername defaultusername -proxyPassword defaultpassword

> unset reputation settings –proxyserver –proxyport -proxyUsername -proxyPassword

> sh reputation settings

Note:

The Proxy Server IP can be an IP address or a fully qualified domain name (FQDN).

Configure IP reputation by using NetScaler GUI

  1. Navigate to the System > Settings. In the Modes and Features section, click the link to access the Configure Advanced Features pane and enable the Reputation checkbox.
  2. Click OK.

To configure a proxy server by using the NetScaler GUI

  1. On the Configuration tab, navigate to Security > Reputation.
  2. Under Settings, click Change Reputation Settings to configure a proxy server.
  3. Enable or disable the reputation feature.
  4. Enter the following details to configure the proxy server:
    1. Proxy Server - It can be an IP address or a fully qualified domain name (FQDN).
    2. Proxy port - It accepts values between [1–65535].
    3. Proxy Username – Provide a user name for proxy server authentication.
    4. Proxy Password - Provide a password for proxy server authentication.

      Note: The proxyUsername and proxyPassword fields are enabled if the proxyServer and proxyPort fields are configured.

Create an allow list and a block list of client IP addresses using the GUI

  1. On the Configuration tab, navigate to AppExpert > Data Sets.
  2. Click Add.
  • In the Create Data Set (or Configure Data set) pane, provide a meaningful name for the list of the IP addresses. The name must reflect the purpose of the list.
  • Select Type as IPv4 or IPv6.

  • Click Insert to add an entry.

  • In the Configure Policy dataset binding pane, add an IPv4 or IPv6 format IP address in the Value input box.
  • Provide an index.
  • Add a comment that explains the purpose of the list. This step is optional, but is recommended because a descriptive comment is helpful in managing the list.

Similarly, you can create a block list and add the IP addresses that are to be considered malicious.

Also see, Pattern sets and data sets for more details regarding using data sets and configuring Advanced policy expressions.

Configure an application firewall policy by using the NetScaler GUI

  1. On the Configuration tab, navigate to Security > Application Firewall > Policies > Firewall. Click Add to add a policy using the PI expression to use IP reputation.

You can also use the Expression editor to build your own policy expression. The list shows preconfigured options that are useful for configuring an expression using the threat categories.

Highlights

  • Quickly and accurately stop bad traffic at the network’s edge from known malicious IP addresses posing different types of threats. You can block the request without parsing the body.
  • Dynamically configure IP reputation functionality for multiple applications.
  • Secure your network against data breach without a performance penalty, and consolidate protections onto a single service fabric using fast and easy deployments.
  • You can do IP Reputation checks on source and destination IPs.
  • You can also inspect the headers to detect malicious IPs.
  • IP reputation check is supported in both forward proxy and reverse proxy deployments.
  • The IP Reputation process connects with Webroot and updates the database every 5 minutes.
  • Each node in the High Availability (HA) or Cluster deployment gets the database from Webroot.
  • The IP reputation data is shared across all partitions in admin-partition deployments.
  • You can use an AppExpert data set to create lists of IP addresses to add exceptions for IPs blocklisted in the Webroot database. You can also create your own customized block list to designate specific IPs as malicious.
  • The iprep.db file is created in the /var/nslog/iprep folder. Once created, it is not deleted even if the feature is disabled.
  • When the reputation feature is enabled, the NetScaler Webroot database is downloaded. After that, it is updated every 5 minutes.
  • The Webroot database major version is version: 1.
  • The minor version gets updated every day. The update version is incremented after every 5 minutes and is reset back to 1 when the minor version is incremented.
  • PI expressions enable you to use IP reputation with other features, such as responder and rewrite.
  • The IP addresses in the database are in decimal notation.

Debugging tips

  • If you cannot see the reputation feature in the GUI, verify that you have the right license.
  • Monitor the messages in var/log/iprep.log for debugging.
  • Webroot connectivity: If you see the ns iprep: Not able to connect/resolve WebRoot message, make sure that the appliance has internet access and DNS is configured.
  • Proxy server: If you see the ns iprep: iprep_curl_download: 88 curl_easy_perform failed. Error code: 5 Err msg:couldnt resolve proxy name message, make sure that the proxy server configuration is accurate.
  • IP Reputation feature not working: The IP Reputation process takes about five minutes to start after you enable the reputation feature. The IP reputation feature might not work for that duration.
  • Database download: If the IP DB data download is failing after enabling the IP Reputation feature, the following error is seen in the logs.

iprep: iprep_curl_download:86 curl_easy_perform failed. Error code:7 Err msg:Couldn't connect to server

Solution: Allow the out-bound traffic to the following URLs or configure a proxy to resolve the issue.

localdb-ip-daily.brightcloud.com:443
localdb-ip-rtu.brightcloud.com:443
api.bcti.brightcloud.com:443
localdb-ipv6-daily.brightcloud.com:443
ipce-daily.brightcloud.com:443
ipce-rtu.brightcloud.com:443
<!--NeedCopy-->
IP Reputation
October 30, 2023
Contributed by: 
C S
October 30, 2023
Contributed by: 
C S

In this article

Site feedback | Your privacy choices footer linkYour Privacy Choices | Privacy and legal terms | Cookie preferences | Consent settings | docs.cloud.com
© 1999- Cloud Software Group, Inc. All rights reserved.