Notes from the lab: Citrix ADC IP Reputation

I’ve been playing around with the Citrix ADC IP Reputation feature – https://docs.citrix.com/en-us/citrix-adc/13/reputation/ip-reputation.html in the lab for some time and to be honest it’s such a small but very effective feature which I almost never see active, why is that?

If you’ve gotten a premium licensed ADC appliance it’s a simple right click>enable and you put in the necessary arguments in a responder policy. See the following article for a quick how to video – https://www.youtube.com/watch?v=WedxwiEVuG4 and basically that is it. The requests are going to be filtered on a Webroot service provider for malicious IP database and you can then drop those from ever getting at you network. (and put in a nifty log action so that you can filter as a syslog entry in Citrix ADM

I put a global responder in place with the expression: CLIENT.IP.SRC.IPREP_IS_MALICIOUS and a reset with accompanied log entry: CLIENT.IP.SRC  + ” connection was dropped by Responder Action for malicious IP when accessing ” + HTTP.REQ.URL the results were pretty much mind blowing, see the following screenshot:Since the exploit CVE-2019-19781 – Vulnerability in Citrix Application Delivery Controller, Citrix Gateway, and Citrix SD-WAN WANOP appliance – https://support.citrix.com/article/CTX267027 the focus was pretty much around that but with the above rule in place I got more hits on the reputation feature than on the mitigation responder in that time. (And these are probes/attacks we don’t know about!)

So to sum it up this is a very good starting point if something like Application Firewall is a bridge to far but most definitely will improve your security with a simple setup.

Hope it helps.

 

 

Notes from the lab: Citrix ADC Native Push OTP not working

I’ve updated my lab environment with Citrix Gateway push OTP support and had some trouble in configuring the Citrix SSO app on my iPhone. For some reason it couldn’t setup the gateway connection and it wasn’t reachable. (Well that was my bad in checking all my devices but I’ll get to that)

Before the push OTP change I’ve worked with the authenticator app behavior and put in the code myself and this worked fine. The change to push OTP isn’t too difficult, and the following articles give you plenty of how-to information:
https://docs.citrix.com/en-us/citrix-gateway/13/push-notification-otp.html
https://docs.citrix.com/en-us/citrix-adc/13/aaa-tm/native-otp-authentication/otp-encryption-tool.html
https://www.carlstalhood.com/native-one-time-passwords-otp-citrix-gateway-13/

Keep in mind that if you have setup a previous OTP setup like I had the encryption part needs to be migrated otherwise it just won’t work. Registration cannot complete if you just flip it on and be done with it. Follow the migration part or just start fresh with OTP encryption enabled by default.

After I corrected the encryption problem I still could not register through the Citrix SSO app for push OTP functionality. It kept giving me the error gateway not reachable, fully checked everything in my environment was up and working. It kept me puzzling that it works fine with the previous authenticator method and if I re-registered it that way it would work (without the push of course because that functionality sits in the Citrix SSO app)

Finally after more troubleshooting I found the problem… because I’ve upgraded and integrated ADFS 2019 in my environment my content switching server and gateway etc. also needed to be SNI aware. Remember that everything worked fine on my Windows devices (even the Citrix SSO VPN functionality which I use quite often) but just not on my iPhone. Turning off SNI was the solution, it seems that the Citrix SSO app on iOS doesn’t support SNI.

Hope it helps!

Notes from the field: Cannot access Citrix ADC or create HA set

Quite recently I was at a customer where they had an SDX setup with single instances and needed to be upgraded and converted to an HA setup.

Well easy does it I created the instances on the second SDX and started creating HA sets. Numerous went fine and then one started giving errors. Could not propagate from the primary and after checking SSH/SCP access this would fail as well. I logged in through the console of SDX/SVM and saw that the sshd daemon wasn’t starting anymore. (On a side note all of the original SDX instances were upgraded in regard to the exploit of last December)

After some troubleshooting I came across the following discussion article: https://discussions.citrix.com/topic/405628-unable-to-connect-to-adc-nsip-version-121-and-130-using-sshsftp

The discussion referred to an support article regarding false positives and an SSH vulnerability:
https://support.citrix.com/article/CTX209398

After checking the sshd_config file and commenting out the following:
#option UsePrivilegeSeparation
#MACs hmac-sha1,hmac-ripemd160

The sshd daemon started again and the HA propagation and synchronisation started instantly. I’ve had this on several other instances as well and they all needed the above commenting out of the lines.

Hope it helps!

Notes from the lab: Citrix ADC Native OTP and AdminSDHolder

While doing some lab work I came across an issue that the Domain Admin accounts could not register on the manageotp site while Domain Users could. This got me figuring it out.

For the use of Native OTP on the ADC we need to use an bind account for Active Directory which has the appropriate write permissions on the userParameters value of the users.

When we delegate control of the exact write permission of the userParameters everything is fine for normal users but administrator accounts won’t work. When we use a service account with full blown domain administrator permissions as the bind account then it works.

After some researching I came across this old article which explained the behavior:
https://support.microsoft.com/nl-nl/help/817433/delegated-permissions-are-not-available-and-inheritance-is-automatical

Long story short, if any user is also a member of a high privileged group the AdminSDHolder protection will prevent this. There is a way that inheritance can be enabled but this is mostly not recommended as you will open up a whole lot of extra security risks.

If it isn’t needed then just delegate control of the needed permissions otherwise use an bind account with domain admin permissions.

For some in depth knowledge of AdminSDHolder and it’s workings see the following article:
https://www.petri.com/active-directory-security-understanding-adminsdholder-object

Notes from the field: Citrix ADC Gateway Native OTP with GSLB

Fun quick fact that I’ve encountered when deploying a ADC Gateway GSLB setup for a customer! You only have to enroll once with the nFactor/Native OTP on one of the ADC’s. (when having a Active Directory Domain across multiple datacenter sites)

The setup of choice:

  • Two ADC appliances in HA set on each site
  • GSLB enabled in active/passive mode for the Gateway across both sites
  • Native OTP enabled and active as the way for authentication
  • Active Directory Domain across two sites

There is no difference in configuration whatsoever because the magic of Native OTP depends on Active Directory.

Configure each ADC identically with the nFactor/Native OTP setup and enable GSLB and you’re done. I must admit at first I thought that I would need to enroll at both gateways independent but happily this is not the case.

For the configuration steps see common examples as below:

https://docs.citrix.com/en-us/netscaler-gateway/12-1/native-otp-support.html

https://www.carlstalhood.com/netscaler-gateway-12-native-one-time-passwords-otp/