Use jobs to upgrade NetScaler instances

You can use NetScaler Console to upgrade one or more NetScaler instances. You must know the licensing framework and types of licenses before you upgrade an instance.

NOTE: If you want to upgrade an instance that has classic policies, we recommend that you convert the classic policies to advanced policies before upgrading the instance, using the NSPEPI tool. This is applicable for the features that are supported by the NSPEPI tool. For more information, see Upgrade considerations for configurations with classic policies.

When you upgrade your NetScaler instance by creating a maintenance job, perform the pre-validation check on the instances that you want to upgrade.

Pre-validation checks

1. Check for customizations - Back up your customizations and delete them from the instances. You can reapply the backed-up customizations after the instance upgrade.

2. Check the disk usage - If the /var folder has less than 6 GB space and the /flash folder has less than 200 MB space, clean up the disk space. Check the following folder paths to clean the disk space:

   • /var/nstrace
   • /var/log
   • /var/nslog
   • /var/tmp/support
   • /var/core
   • /var/crash
   • /var/nsinstall
   • /var/netscaler/nsbackup

3. Check for disk hardware issues - Resolve the hardware issues if any.

4. Check for STAYPRIMARY and STAYSECONDARY nodes - For a NetScaler HA, the upgrade is blocked for the nodes in STAYPRIMARY and STAYSECONDARY states. These nodes are identified in the pre-validation check and listed under Instances blocked from upgrade.

NetScaler high-availability pair

When you upgrade a NetScaler HA pair, note the following:

• The secondary node is upgraded first.

• Synchronization and propagation of the nodes are disabled until both nodes are upgraded successfully.

• After the successful HA pair upgrade, an error message appears in the execution history. This message appears if your nodes in the HA pair are on different builds or versions. This message indicates that synchronization between primary and secondary node is disabled.

You can upgrade a NetScaler HA pair in two stages:

1. Create an upgrade job and run on one of the nodes immediately or schedule it later.

2. Schedule the upgrade job to run on the remaining node later. Ensure to schedule this job after the initial node’s upgrade.

NetScaler clusters

When you upgrade a NetScaler cluster, the NetScaler Console does pre-upgrade validation on the specified instance only. Before you upgrade, check and resolve customization, disk usage, and hardware issues on the cluster nodes.

Create an upgrade maintenance job to upgrade NetScaler instances

Note

NetScaler upgrade from a higher version to a lower version is not supported. For example, if your NetScaler instance is 13.0 82.x, you cannot downgrade the NetScaler instance to 13.0 79.x or any other earlier versions.

1. In NetScaler Console, navigate to Infrastructure > Upgrade Jobs. Click the Create Job button.

2. In Create Maintenance Jobs, select Upgrade NetScaler (Standalone/High-Availability/Cluster) and click Proceed.

   

3. In Select Instance, type a name of your choice for Job Name.

4. Click Add Instances to add NetScaler instances that you want to upgrade.

   • To upgrade a NetScaler high-availability pair, select the IP addresses of the high-availability pair (denoted by the superscript of ‘S’ and ‘P’).

   • To upgrade a cluster, select the cluster IP address (denoted by the superscript of ‘C’).

   

5. In the Select Image tab, select an NetScaler image from your local drive or the build images.

   • Local - Select the instance upgrade file from your local machine.

   • Appliance - Select the instance upgrade file from a NetScaler Console file browser. The NetScaler Console GUI displays the instance files that are present at /var/mps/ns_images.

   

   

   • Skip image uploading to NetScaler if the selected image is already available - This option checks whether the selected image is available in NetScaler. Upgrade job skips uploading a new image and uses the image available in NetScaler.

   • Clean software image from NetScaler on successful upgrade - This option clears the uploaded image in the NetScaler instance after the instance upgrade.

   Click Next to start the pre-upgrade validation on the selected instances.

   Note:

   • The downloaded NetScaler images are stored in the NetScaler Console agent and are present in /var/mps/adcimages. These cached images can be used for multiple NetScaler upgrades, thus eliminating the need to download an image each time for an upgrade.
   • NetScaler Console clears the cached NetScaler images every three days based on the last modified time of the images. Only the latest two image files are cached in the NetScaler Console agent at a time.

6. The Pre-upgrade validation tab displays the following sections:

   • Instances ready for upgrade. You can continue with the upgrade of these instances.

   • Instances blocked from upgrade. These NetScaler instances are blocked from upgrade because of pre-upgrade validation errors.

     You can review, rectify the errors, and then click Move to ready for upgrade to upgrade them. If you face insufficient disk space on an instance, you can check and clean up the disk space. See, Clean up NetScaler disk space.

   

   • Policy Check: If NetScaler Console finds unsupported classic policies, you can remove such policies to create an upgrade job.

   Important:

   If you specify cluster IP address, the NetScaler Console does pre-upgrade validation only on the specified instance not on the other cluster nodes.

   To view discrepancies between primary and secondary nodes during an upgrade, select the high-availability node, and click Details.

   

   

   • Configuration discrepancies found in primary node of HA - Displays all the configurations found in the secondary node of the NetScaler high-availability pair but missing in the primary node.

   • Configuration discrepancies found in secondary node of HA - Displays all the configurations found in the primary node of the NetScaler high-availability pair but missing in the secondary node.

   Note:

   You can ignore the following discrepancies that may appear in the configuration discrepancies sections:

   • Device-specific configurations like IP addresses.

   • Encrypted passwords or certificates, which may differ between nodes, even if the password is the same.

   You can review the discrepancies and choose to ignore them if they are not relevant.

7. In Validation Scripts, specify the scripts to run before and after an instance upgrade. You can do either of the following:

   • Default Validation Scripts - Choose this option to run the predefined validation scripts. These scripts are run both before and after the upgrade job, generating a difference report for the validation script.

     Note:

     You cannot change or edit this predefined set of commands.

   • Custom Validation Scripts - Choose this option to run your validation script. You can specify if you want the scripts to be run before or after the upgrade. A diff report is generated only if the same scripts are selected before and after the upgrade.

   To know the set of commands in each configuration, click View Details.

   

   In Custom Validation scripts, specify the scripts to run before and after an instance upgrade. Use one of the following ways to run the commands:

   The custom scripts are used to check the changes before and after an NetScaler instance upgrade. For example:

   • The instance version before and after the upgrade.
   • The status of interfaces, high-availability nodes, virtual servers, and services before and after upgrade.
   • The statistics of virtual servers and services.
   • The dynamic routes.

   An instance upgrade has multiple stages. You can now specify these scripts to run in the following stages:

   • Pre upgrade: The specified script runs before upgrading an instance.

   • Post upgrade pre failover (applicable for HA): This stage only applies to the high-availability deployment. The specified script runs after upgrading the nodes, but before their failover.

   • Post upgrade (applicable for standalone) / Post upgrade post failover (applicable for HA): The specified script runs after upgrading an instance in the standalone deployment. In the high-availability deployment, the script runs after upgrading the nodes and their failover.

   Note:

   Ensure to enable script execution at the required stages. Otherwise, the specified scripts do not run.

   You can import a script file or type commands directly in the NetScaler Console GUI.

   • Use same script as Pre upgrade: Use the same custom script for pre-upgrade, pre-failover, and post-upgrade.
   • Import commands from file: Select the command input file from your local computer.
   • Type commands: Enter the commands directly on the GUI.

8. In Schedule Task, select one of the following options:

   • Upgrade now - The upgrade job runs immediately.

   • Schedule Later - Select this option to run this upgrade job later. Specify the Execution Date and Start Time when you want to upgrade the instances.

     If you want to upgrade a NetScaler HA pair in two stages, select Perform two stage upgrade for nodes in HA.

     Specify the Execution Date and Start Time when you want to upgrade another instance in the HA pair.

     

9. In Create Job, specify the following details:

   1. Select one of the following options from the Software Image list:

      • Local - Select the instance upgrade file from your local machine.

      • Appliance - Select the instance upgrade file from a file browser. The NetScaler Console GUI displays the instance files that are present at /var/mps/mps_images.

   2. Specify when you want to upload the image to an instance:

      • Upload now - Select this option to upload the image immediately. However, the upgrade job runs at the scheduled time.

      • Upload at the time of execution - Select this option to upload the image at the time of upgrade job execution.

      For a high-availability pair, you can specify the nodes on which you want to upload the image:

      • Upload to both primary and secondary nodes: Upload the build image file to both the primary and secondary nodes.

      • Upload to secondary node only: Upload the build image file to only the secondary node. After the secondary node is upgraded, a failover occurs and the build image file is uploaded to the new secondary node which was previously, the primary node.

   

   For more information on the available scheduling scenarios for high-availability pair, see Scheduling upgrade jobs for high-availability pair.

   • Clean software image from NetScaler on successful upgrade - Select this option to clear the uploaded image in the NetScaler instance after the instance upgrade.

   • Backup the NetScaler instances before starting the upgrade. - Creates a backup of the selected NetScaler instances.

   • Maintain the primary and secondary status of HA nodes after upgrade: Select this option if you want the upgrade job to start a failover after each node’s upgrade. In this way, the upgrade job maintains the primary and secondary status of the nodes.

   • Save NetScaler configuration before starting the upgrade - Saves the running NetScaler configuration before upgrading the NetScaler instances.

   • Enable ISSU to avoid network outage on NetScaler HA pair - ISSU ensures the zero downtime upgrade on an NetScaler high-availability pair. This option provides a migration functionality that honors the existing connections during the upgrade. So, you can upgrade a NetScaler HA pair without downtime. Specify the ISSU migration timeout in minutes.

   • Receive Execution Report through email - Sends the execution report in email. To add an email distribution list, see Create an email distribution list.

   • Receive Execution Report through slack - Sends the execution report in slack. To add a Slack profile, see Create a Slack profile.

   Click Create Job.

The upgrade job appears in the Infrastructure > Upgrade Jobs. When you edit an existing job, you can switch to any tabs if the required fields are already filled. For example, if you are in the Select Configuration tab, you can switch to the Job Preview tab.

Pause or resume a scheduled upgrade job

You can also pause your scheduled upgrade job.

To use this feature, navigate to Infrastructure > Upgrade Jobs, select an existing scheduled upgrade job, and click Stop to pause the job. To resume the scheduled upgrade job, click Resume.

Note:

If the scheduled time for the upgrade job has passed after you decided to resume it, you need to create the upgrade job again.

Retry failed upgrade jobs

1. In Infrastructure > Upgrade Jobs, select the failed upgrade job and click Retry. Alternatively, you can also go to Select Action > Retry Upgrade Job to retry a failed job.

   

2. In Select Instance, specify the following details:

   • Job Name - Enter a name for the upgrade.

   • Select the NetScaler instances that you want to upgrade from the list. To delete any instances, click Remove.

   Click Next to begin the validation process.

   

3. The Pre-upgrade validation tab displays the following sections:

   • Instances ready for upgrade. You can continue with the upgrade of these instances.

   • Instances blocked from upgrade. These NetScaler instances are blocked from upgrade because of pre-upgrade validation errors.

     You can review, rectify the errors, and then click Move to ready for upgrade to upgrade them. If you face insufficient disk space on an instance, you can check and clean up the disk space. See, Clean up the NetScaler disk space.

   • Policy Check: If NetScaler Console finds unsupported classic policies, you can remove such policies to create an upgrade job.

   

   Click Next.

4. In Schedule Task, select one of the following options:

   • Upgrade now: The upgrade job runs immediately.

   • Schedule Later: Select this option to run this upgrade job later. Specify the Execution Date and Start Time when you want to upgrade the instances.

   

   Click Retry.

Clean up the NetScaler disk space

If you face the insufficient disk space issue while upgrading an NetScaler instance, clean up the disk space from the NetScaler Console GUI itself.

1. In the Pre-upgrade validation tab, the Instances blocked from upgrade section displays the instances that failed the upgrade because of Insufficient disk space. Select the instance that has the disk space issue.

2. Click Check Disk Space.

   A Disk Space Details page appears. This page displays the instances, used memory, and available memory.

   

3. In the Disk Space Details pane, select the instance that requires clean up and do one of the following:

   1. Disk Cleanup - Navigate to the required folders or directories and delete them to free up disk space.

   2. Quick Cleanup - Quickly clear up disk space by deleting multiple folders. In the Confirm pane that appears, select the folders you want to delete and click Yes.

      

   3. After clearing up the disk space, you can check if sufficient disk space is now available to upgrade the instance. In the Instances blocked from upgrade section, click Revalidate.

      In the following example, disk space is available. You can now click Move to ready for upgrade to upgrade the instance or click Next to continue to the next step.

      

Scheduling upgrade jobs for a NetScaler high-availability pair

The following table lists the different scheduling scenarios in the Schedule Task page and the corresponding upgrade options available in the Create Job page:

Download a combined diff report of a NetScaler upgrade job

You can download a difference report of an NetScaler upgrade job if custom scripts are specified. A diff report contains the differences between the outputs of the pre-upgrade and post-upgrade scripts. With this report, you can determine what changes occurred on the NetScaler instance post upgrade.

Note

The diff report is generated only if you specify the same script in the pre-upgrade and post-upgrade stages.

To download a diff report of an upgrade job, do the following:

1. Navigate to Infrastructure > Upgrade Jobs.

2. Select the upgrade job for which you want to download a diff report.

3. Click Diff Reports.

4. In Diff Reports, download a consolidated diff report of the selected upgrade job.

   In this page, you can download any of the following diff reports:

   • Pre vs Post upgrade pre failover diff report
   • Pre vs Post upgrade diff report