Configure disaster recovery for high availability
Disaster is a sudden disruption of business functions caused by natural calamities or human caused events. Disasters affect data center operations, after which resources and the data lost at the disaster site must be fully rebuilt and restored. The loss of data or downtime in the data center is critical and collapses the business continuity.
The NetScaler ADM disaster recovery (DR) feature provides full system backup and recovery capabilities for NetScaler ADM deployed in high availability mode. At the time of recovery, certificates, configuration files, and a complete backup of the database is available in the recovery site.
The following table describes the terms used while configuring disaster recovery in NetScaler ADM.
| Terms | Description |
|---|---|
| Primary site (Data center A) | The primary site has NetScaler ADM nodes deployed in high availability mode. |
| Recovery site (Data center B) | The recovery site has a disaster recovery node deployed in standalone mode. This node is in read-only mode and is not operational until the primary site is down. |
| Disaster recovery node | The recovery node is a standalone node deployed in the recovery site. This node is made operational (to the new primary) in case a disaster occurs at the primary site and it is nonfunctional. |
Note
The primary site and DR site communicate with each other through ports 5454 and 22, and these ports are enabled by default.
For more information on port and protocol details, see Ports.
Disaster recovery workflow
The following image shows the disaster recovery workflow, the initial setup before disaster, and the workflow after the disaster.
Initial setup before disaster
The image shows the disaster recovery setup before disaster.
The primary site has NetScaler ADM nodes deployed in the high availability mode. To learn more, see High availability deployment
The recovery site has a standalone NetScaler ADM disaster recovery node deployed remotely. The disaster recovery node is in read-only mode and receives data from the primary node to create data backup. NetScaler instances in the recovery site are also discovered but, they do not have any traffic flowing through them. During the backup process, all data, files, and configurations are replicated on the disaster recovery node from the primary node.
Prerequisites
Before you set up the disaster recovery node, note the following the prerequisites:
-
To enable disaster recovery settings, the primary site must have NetScaler ADM nodes configured in high availability mode.
-
The standalone deployment of NetScaler ADM in the primary site does not support the disaster recovery feature.
-
The NetScaler ADM HA pair (in primary site) and the standalone node (in DR site) must have same software version, build, and configurations.
Citrix recommends that you set CPU priority (in virtual machine properties) at the highest level to improve scheduling behavior and network latency.
The following table lists the minimum requirements to configure the Disaster Recovery node:
| Component | Requirement |
|---|---|
| RAM | 32 GB |
| Virtual CPU | 8 CPUs |
| Storage Space | Citrix recommends using solid-state drive (SSD) technology for NetScaler ADM deployments. The default value is 120 GB. Actual storage requirement depends on NetScaler ADM sizing estimation. If your NetScaler ADM storage requirement exceeds 120 GB, you have to attach an extra disk. Note You can add only one more disk. Citrix recommends you to estimate storage and attach more disk at the time of initial deployment. For more information, see How to Attach an Additional Disk to NetScaler ADM. |
| Virtual network interfaces | 1 |
| Throughput | 1 Gbps or 100 Mbps |
| Hypervisor | Versions |
| Citrix Hypervisor | 6.2 and 6.5 |
| VMware ESXi | 5.5 and 6.0 |
| Microsoft Hyper-V | 2012 R2 |
| Linux KVM | Ubuntu and Fedora |
First time disaster recovery setup
-
Deploy NetScaler ADM in high availability mode
-
Deploy and register the NetScaler ADM disaster recovery node
-
Enable and disable disaster recovery settings from the user interface
Deploy NetScaler ADM in high availability mode
To set up the disaster recovery settings, ensure that NetScaler ADM is deployed in high availability mode. For information on deploying the NetScaler ADM in high availability, see High availability deployment
Note
NetScaler ADM deployed in high availability mode must be upgraded to NetScaler ADM release version 13.1.
Floating IP address is mandatory to register disaster recovery node with the primary node.
Deploy and register the NetScaler ADM disaster recovery node using DR console
To register the NetScaler ADM disaster recovery node:
-
Download the
.xvaimage file from the NetScaler site and import it into your hypervisor. -
From the Console tab, configure NetScaler ADM with the initial network configurations.
Note
The disaster recovery node can be on a different subnet.
-
After the initial network configuration is complete, the system prompts for login. Log on using the following credentials –
nsrecover/nsroot.Important
Do not change the DR node credentials (
nsrecover/nsroot) during registration. You can change the DR node credentials after you register DR node successfully. -
To deploy the disaster recovery node, type /mps/deployment_type.py and press enter. The NetScaler ADM deployment configuration menu is displayed.
-
Select 2 to register disaster recovery node.
-
The console prompts for floating IP address of the high availability node and password.
-
Enter the floating IP address and password to register the disaster recovery node to the primary node.
The disaster recovery node is now registered successfully.
Note
-
The disaster recovery node does not have a GUI.
-
After registration is successful, the default administrator credentials to log on to the server are
nsroot/nsroot.
-
-
If you want to change the DR node password, run the following script:
/mps/change_freebsd_password.sh <username> <password> <!--NeedCopy-->Example:
/mps/change_freebsd_password.sh nsroot new_password <!--NeedCopy-->
Deploy the disaster recovery node using NetScaler ADM GUI
After the disaster recovery node is registered successfully using DR console, deploy the DR node from the NetScaler ADM GUI. This step enables the disaster recovery settings from the NetScaler ADM primary site.
-
Navigate to System > System Administration > Disaster Recovery Settings.
-
On the Disaster Recovery page, select Deploy DR Node.
-
A confirmation dialogue box is displayed. Click Yes to continue.
Note
The time taken for system backup depends on the data size and the WAN link speed.
After you deploy the DR node successfully in the NetScaler ADM GUI, you can monitor database state, memory, CPU, and disk usage of the DR node.
To disable the disaster recovery settings, select Remove DR Node. A confirmation dialogue box is displayed. Click Yes to continue.
To enable the DR node again, reconfigure the DR node for your high availability pair:
-
Log on to the DR node using a hypervisor or an SSH console.
-
Configure the DR node, by following the procedure available at Deploy and register the NetScaler ADM disaster recovery node using DR console.
For more information, see the FAQs.
Important
It is the responsibility of the administrator to detect that a disaster has occurred on the primary site.
The disaster recovery workflow is manually initiated by the administrator after the primary site goes down.
An administrator must manually initiate the process by running a recovery script on the disaster recovery node at the recovery site.
If you upgrade the HA pair in primary site, you must also manually upgrade the standalone node in the DR site.
Workflow after the disaster
When the primary site goes down after a disaster, the disaster recovery workflow must be initiated as follows:
-
The administrator identifies that a disaster has struck the primary site and it is not operational.
-
The administrator initiates the recovery process.
-
The administrator must manually run one of the following recovery scripts on the disaster recovery node based on your requirement(at the recovery site):
-
Cofigure SNMP, Syslog, and Analytics on the DR node:
/mps/scripts/pgsql/pgsql\_restore\_remote\_backup.sh <!--NeedCopy--> -
Configure the DR node as a license server also:
/mps/scripts/pgsql/pgsql\_restore\_remote\_backup.sh -reconfig-ls <IP-address-of-the-primary-site> <!--NeedCopy-->
-
-
Internally, NetScaler instances are automatically reconfigured to send the data to the disaster recovery node that has now become the new primary site.
The following image shows that the disaster recovery workflow after the primary site is struck with a disaster.
Note:
After you initiate the script at the DR site, the DR site now becomes the new primary site. You can also access the DR user interface.
Post disaster recovery
After the disaster has occurred and the administrator initiates the recovery script, the DR site now becomes the new primary site.
If you want to revert the configurations to the original site later, see Revert configurations to the original primary site.
Important
If you have installed NetScaler ADM 12.1.49.x or earlier releases, you get a grace period of 30 days to contact Citrix to rehost the original license on the NetScaler ADM (at the DR site).
For 12.1.50.x or later releases, the NetScaler ADM license is automatically synchronized to the DR site (Not a requirement to contact Citrix for the license).
If you have applied pooled licenses for the instances, NetScalers with version 11.1 65.x or later, 12.1 58.x or later, 13.0 47.x or later, and NetScaler SDX 13.0 76.x or later have the support for auto-license server update in the DR site. All other versions, you must manually reconfigure the instances to the DR site.
Revert configurations to the original primary site
Post disaster the configured disaster recovery (DR) node becomes the new primary site and the client traffic flows through this node.
For more information, see Workflow after the disaster.
When your original primary site is free from disaster and you decide to move all operations to the primary site, reconfigure the original primary site to match the configurations from the DR node.
Before you begin, ensure both primary site and DR site are active.
To revert the changes to the original primary site from the DR site, perform the following steps:
-
Log in to the original primary site and run the following command:
nohup /mps/sync_adm_node.py -I <DR-site-IP-address> -R <DR-node-password> -L <primary-node-password> & <!--NeedCopy-->This command configures only Syslog, SNMP, and Analytics to the primary site.
If you want to configure the primary site as a pooled license server for ADC instances, run the following command:
nohup /mps/sync_adm_node.py -I <DR-site-IP-address> -R <DR-node-password> -L <primary-node-password> -O yes & <!--NeedCopy-->The
-Ocommand fetches the DR site IP address and reconfigures the primary site as pooled license server. -
Reconfigure the DR site. See, Deploy disaster recovery setup.
After you successfully revert the configurations from the DR site to the original primary site, the client traffic flows through the NetScaler ADM primary node.