ADC

High CPU

September 14, 2021

Contributed by:

S C

Following are some of the functionality and high CPU related debugging issues encoutered and the best practices to follow when working with Web App Firewall:

Check Policy hits, Bindings, Network configuration, Web App Firewall configuration:

Identify misconfiguration
Identify vserver that is serving the affected traffic

Inspect logs in the following log files for security violations and recent configuration changes:

/var/log/ns.log
/var/nslog/import.log
/var/nslog/aslearn.log
tail -f /var/log/ns.log | grep APPFW_SIGNATURE_MATCH

Example:

Jun 13 01:11:09 <local0.info> 10.217.31.98 CEF:0|Citrix|NetScaler|NS11.0|APPFW| APPFW_SIGNATURE_MATCH|6|src=10.217.253.62 spt=61141 method=GET request= http://aaron.stratum8.net/FFC/wwwboard/passwd.txt msg=Signature violation rule ID 807: web-cgi /wwwboard/passwd.txt access cn1=140 cn2=841 cs1=pr_ffc cs2=PPE0 cs3=OyTgjbXBqcpBFeENKDlde3OkMQ00001 cs4=ALERT cs5=2015 cs6=web-cgi act=not blocked
<!--NeedCopy-->

Isolate the traffic that is effected:

Isolate the profile
Isolate the security check
Isolate the URL, virtual server and traffic parameters

Conditional profile level trace helps identify the traffic and violation records:

set appfw profile <profile> -trace ON
start nstrace -mode APPFW -size 0
stop nstrace

Note: Ensure that the trace is collected with -size 0 option.

Check appfw, dht, IP reputation activity counters:

nsconmsg -g as_ -g appfwreq_ -g iprep -d current

Monitor window size for resets in connection:

Appfw sets the window size to 9845 when Citrix ADC resets the connection due to an invalid http message.

Examples:

Malformed request received - connection reset
High CPU related issues
Check data sheets for system limits
Inspect for cpu usage, appfw, DHT and memory related activity. Monitor appfw sessions
nsconmsg -g cc_cpu_use -g appfwreq -g as -g dht -g mem_AS_OBJ -g mem_AS_COMPONENT -d current

Monitor memory allocated and freed from Web App Firewall components and objects during the target time period. It helps in isolating the protection leading to high CPU usage.

Profiler output
Observe logs

Isolate appfw check leading to high CPU:

startURLClosure
Formfiledconsistency
CSRF
Cookie protections
Referer header check

Ascertain that autoupdate of signatures is not leading to high CPU (Disable to confirm).

The official version of this content is in English. Some of the Cloud Software Group documentation content is machine translated for your convenience only. Cloud Software Group has no control over machine-translated content, which may contain errors, inaccuracies or unsuitable language. No warranty of any kind, either expressed or implied, is made as to the accuracy, reliability, suitability, or correctness of any translations made from the English original into any other language, or that your Cloud Software Group product or service conforms to any machine translated content, and any warranty provided under the applicable end user license agreement or terms of service, or any other agreement with Cloud Software Group, that the product or service conforms with any documentation shall not apply to the extent that such documentation has been machine translated. Cloud Software Group will not be held responsible for any damage or issues that may arise from using machine-translated content.

High CPU

September 14, 2021

Contributed by:

S C

September 14, 2021

Contributed by:

S C

High CPU

In this article