ADC

Bot signature auto update

The bot static signature technique uses a signature lookup table with a list of good and bad bots. The bots are categorized based on the user-agent string and domain names. If the user-agent string and domain name in incoming bot traffic matches a value in the lookup table, the configured bot action is applied. The bot signature updates are hosted on the AWS cloud and the signature lookup table communicates with the AWS database for signature updates. The auto signature update scheduler runs every 1-hour to check the AWS database and updates the signature table in NetScaler.

The signature auto update URL to configure is, https://nsbotsignatures.s3.amazonaws.com/BotSignatureMapping.json

Note:

You can also configure a proxy server and periodically update signatures from the AWS cloud to the appliance through the proxy. For proxy configuration, you must set the proxy IP address and port address in the bot settings.

How bot signature auto update works

The following diagram shows how the bot signatures are retrieved from the AWS cloud, updated on NetScaler, and viewed on NetScaler Console for signature update summary.

Bot signature auto update

The bot signature auto-update scheduler does the following:

  1. Retrieves the mapping file from the AWS URI.
  2. Checks the latest signatures in the mapping file with the existing signatures in the ADC appliance.
  3. Downloads the new signatures from AWS and verifies the signature integrity.
  4. Updates the existing bot signatures with the new signatures in the bot signature file.
  5. Generates an SNMP alert and sends the signature update summary to NetScaler Console.

Notes:

Signatures are updated every one hour. For details regarding a specific signature, refer to its associated alert article.

Configure bot signature auto update

For configuring bot signature auto update, complete the following steps:

Enable bot signature auto update

You must enable the auto update option in the bot settings on NetScaler. At the command prompt, type:

set bot settings –signatureAutoUpdate ON

Configure proxy server settings (optional)

If you are accessing the AWS signature database through a proxy server, you must configure the proxy server and port. set bot settings –proxyserver –proxyport

Example:

set bot settings –proxy server 1.1.1.1 –proxyport 1356

Configure bot signature auto update using the NetScaler GUI

Complete the following steps to configure bot signature auto update:

  1. Navigate to Security > NetScaler Bot Management.
  2. In the details pane, under Settings click Change NetScaler Bot Management Settings.
  3. In the Configure NetScaler Bot Management Settings, select the Auto Update Signature checkbox.
  4. Click OK and Close.

Note:

In addition to enabling the bot signature auto-update feature, you must run the command /var/python/bin/python /netscaler/auto_update_bot_signatures.py from the shell prompt to automatically update the bot signatures on NetScaler.

Import default signatures without internet access

It is recommended to configure a proxy server to point to the Amazon (AWS) server to get the latest updates. However, if NetScaler does not have an internet connection to the external sites, then the user can store the updated BOT signature files on a local server. NetScaler can then download the signatures from the local server. In this scenario, the user must constantly check the Amazon site to get the latest updates. You can download and verify the signature file against the corresponding sha1 file which was created by using the Citrix public key to protect against tampering.

To copy the Signature files to a local server, complete the following procedure:

  1. Create a local directory such as <MySignatures> on a local server.
  2. Open the AWS site.
  3. Copy the BotSignaturesMapping.json file file to the <MySignatures> folder.

    If you open the BotSignaturesMapping.json file file, you can see all the json files for signatures and their corresponding digest files for different supported versions. One such pair is highlighted the following image. Sample Pair

  4. Create a subdirectory <botsigs> in the <MySignatures> folder.
  5. Copy all pairs of the files listed in the <file> attribute and the digest files listed in the corresponding digest attribute of the BotSignaturesMapping.json file to the <botsigs> folder. The following are a few sample files that is copied to the <botsigs> folder:

    https://nsbotsignatures.s3.amazonaws.com/BotSignatureMapping.json https://nsbotsignatures.s3.amazonaws.com/botsigs/sig-r14.1b0v17s2 https://nsbotsignatures.s3.amazonaws.com/botsigs/sig-r14.1b0v17s2.digest
  6. Change the auto update URL in the Bot settings in NetScaler to https://<local server>/MySignatures/BotSignatureMapping.json

Note:

You can give any name to the <BOTMySignatures> folder and it can be in any location but the subdirectory <botsigs> must be a subdirectory in the <MySignatures> folder where the mapping file is copied. In addition, ensure that as shown in the BotSignaturesMapping.json, the subdirectory name <botsigs> must have the exact name and is case sensitive. All Signature files and their corresponding sha1 files should be copied under this <botsigs> directory.