Notes from the field: Cannot access Citrix ADC or create HA set

Quite recently I was at a customer where they had an SDX setup with single instances and needed to be upgraded and converted to an HA setup.

Well easy does it I created the instances on the second SDX and started creating HA sets. Numerous went fine and then one started giving errors. Could not propagate from the primary and after checking SSH/SCP access this would fail as well. I logged in through the console of SDX/SVM and saw that the sshd daemon wasn’t starting anymore. (On a side note all of the original SDX instances were upgraded in regard to the exploit of last December)

After some troubleshooting I came across the following discussion article: https://discussions.citrix.com/topic/405628-unable-to-connect-to-adc-nsip-version-121-and-130-using-sshsftp

The discussion referred to an support article regarding false positives and an SSH vulnerability:
https://support.citrix.com/article/CTX209398

After checking the sshd_config file and commenting out the following:
#option UsePrivilegeSeparation
#MACs hmac-sha1,hmac-ripemd160

The sshd daemon started again and the HA propagation and synchronisation started instantly. I’ve had this on several other instances as well and they all needed the above commenting out of the lines.

Hope it helps!

Notes from the lab: Citrix ADC Native OTP and AdminSDHolder

While doing some lab work I came across an issue that the Domain Admin accounts could not register on the manageotp site while Domain Users could. This got me figuring it out.

For the use of Native OTP on the ADC we need to use an bind account for Active Directory which has the appropriate write permissions on the userParameters value of the users.

When we delegate control of the exact write permission of the userParameters everything is fine for normal users but administrator accounts won’t work. When we use a service account with full blown domain administrator permissions as the bind account then it works.

After some researching I came across this old article which explained the behavior:
https://support.microsoft.com/nl-nl/help/817433/delegated-permissions-are-not-available-and-inheritance-is-automatical

Long story short, if any user is also a member of a high privileged group the AdminSDHolder protection will prevent this. There is a way that inheritance can be enabled but this is mostly not recommended as you will open up a whole lot of extra security risks.

If it isn’t needed then just delegate control of the needed permissions otherwise use an bind account with domain admin permissions.

For some in depth knowledge of AdminSDHolder and it’s workings see the following article:
https://www.petri.com/active-directory-security-understanding-adminsdholder-object

Notes from the field: Citrix ADC Gateway Native OTP with GSLB

Fun quick fact that I’ve encountered when deploying a ADC Gateway GSLB setup for a customer! You only have to enroll once with the nFactor/Native OTP on one of the ADC’s. (when having a Active Directory Domain across multiple datacenter sites)

The setup of choice:

  • Two ADC appliances in HA set on each site
  • GSLB enabled in active/passive mode for the Gateway across both sites
  • Native OTP enabled and active as the way for authentication
  • Active Directory Domain across two sites

There is no difference in configuration whatsoever because the magic of Native OTP depends on Active Directory.

Configure each ADC identically with the nFactor/Native OTP setup and enable GSLB and you’re done. I must admit at first I thought that I would need to enroll at both gateways independent but happily this is not the case.

For the configuration steps see common examples as below:

https://docs.citrix.com/en-us/netscaler-gateway/12-1/native-otp-support.html

https://www.carlstalhood.com/netscaler-gateway-12-native-one-time-passwords-otp/