Notes from the field: Citrix FAS request not supported

On a recent Citrix FAS deployment I’ve encountered the following error: “Request not supported” when logging in to a published application or desktop.
Article https://support.citrix.com/article/CTX218941 explains that re-enrollment of the domain controller authentication template or another custom template for Kerberos usage should resolve the error.

A little bit of a background on the environment, an already working Microsoft ADCS environment was in play and in use for other services. From a design/security perspective it was designed that two dedicated Microsoft ADCS servers would be used and two Citrix FAS servers connecting these new servers. The setup was working as expected but only above error would keep coming when trying to access an application or desktop.

We tried re-enrolling the domain controller authentication certificate and this didn’t do the trick, then we decided to let the Domain Controllers get the certificate from the new dedicated Microsoft ADCS servers for Citrix FAS and this did do the trick but with a side effect the chain is changed and other services would be negatively impacted so a rollback was needed.

With this information a Microsoft support case was created and ultimately they confirmed that what is mentioned in the Citrix support article should do the trick. Ok we got confirmation and yes it indeed does work when using the new ADCS servers but the issue of the original ADCS environment was still a mystery.

So next up we decided to repoint the Citrix FAS servers to the existing Microsoft ADCS server to root out any chain or other issues that might be in play. The result was exactly the same and a not supported request as the end result.

Digging deeper in the Microsoft ADCS environment it was after checking the “NTAuthCertificates” store that the existing server wasn’t there and the new servers were. This explained the smartcard logon not working when using the existing environment because an requirement for smartcard logon is that the “NTAuthCertificates” store has the issuing certificate authority propagated. After adding the certificate and waiting for replication and a reboot everything was working as expected, also when moving to the new Microsoft ADCS environment for certificate issuing.

See the following screenshot of the Enterprise PKI snap in MMC in which you can check and/or add the missing certificate:

See the following articles for extra information:
https://docs.microsoft.com/en-us/troubleshoot/windows-server/windows-security/import-third-party-ca-to-enterprise-ntauth-store
https://support.microsoft.com/en-za/help/281245/guidelines-for-enabling-smart-card-logon-with-third-party-certificatio

Notes from the lab: Microsoft ADFS and VMware UAG

You don’t see many configuration articles around ADFS and UAG and that’s why I would like to share my setup.

First things first, I’m expecting that there is an working Horizon environment with True SSO enabled for access to the desktop. And a working ADFS environment to add a new application to test with.

My setup:
1 x ADFS for internal usage
1 x WAP for external usage
1 x UAG v3.10 – dedicated for ADFS with its own URL
1 x UAG v3.10 – dedicated for WS1 with its own URL
2 x Horizon Connection Servers
2 x Horizon Enrollment Servers

The following articles helped me in setting this up:
https://docs.vmware.com/en/Unified-Access-Gateway/3.10/com.vmware.uag-310-deploy-config.doc/GUID-E4C8B88F-C771-4829-ABBE-12F7FBF517C3.html
https://communities.vmware.com/thread/625006
https://thevirtualhorizon.com/2019/12/14/integrating-microsoft-azure-mfa-with-vmware-unified-access-gateway-3-8/
https://techzone.vmware.com/enabling-saml-20-authentication-horizon-unified-access-gateway-and-okta-vmware-horizon-operational

Setting this up starts with downloading the “/FederationMetadata/2007-06/FederationMetadata.xml” file of your ADFS setup and saving this on your configuration machine

Now go to the UAG admin appliance on the management port https://xxx:9443 and scroll down to the Identity Bridging Settings, select the gear icon of “Upload Identity Provider Metadata” and in the next screen press on the select link where you can upload the metadata file of ADFS.
Now go to the Horizon settings on the top and select the gearbox and in the next screen select more to show the auth methods to be used. For True SSO select SAML. Next select the uploaded Identity Provider which is now visible and click on the “Download SAML service provider metadata”. Save everything and close the admin portal.

Next in ADFS configure a new “Relying Party Trust” claims aware application and import the downloaded SAML service provider metadata file from UAG
When completed edit the “Claim issuance policy” for the application and create a rule with attribute store active directory selected and provide the input for “User-Principal-Name” with “Name ID” as outgoing claim type.
Edit the application and select encryption and click on remove the encryption certificate. This enabled will not give an valid SAML assertion or logon.

This is all that is needed in ADFS, the application can be assigned anyway you want. The last steps are in Horizon itself.

Go to the admin panel of the connection server and configure an SAML 2.0 Authenticator, create one and name it accordingly (don’t forget to enable True SSO on this connector) and make sure it is an static type. Copy paste the federationmetadata.xml content into the SAML Metadata screen and click OK.

Set the delegation of authentication to VMware Horizon (SAML 2.0 Authenticator) to optional or required. The latter prohibits passthrough authentication straight from the servers itself and will always require a SAML assertion.

This is it, afterwards you should see an ADFS logon possible with SSO in place.

Some extra information for any use case with ADFS/Workspace ONE combined then this procedure will not help. For Workspace ONE integration the only working and supported way is to configure VMware Horizon with VMware Access. I’ve tried it configured with VMware Access and the same UAG and you will get an access denied because the SAML configuration is in place at the Horizon Connection Servers instead of the UAG. ADFS can also be integrated with VMware Access and the SSO can be achieved in that way which is a route you would take when using Workspace ONE.

Hope it helps!

Notes from the field: Citrix FAS SSO not working with invalid CRL

Recently I got contacted by a customer who had problems performing an SSO to a newly build desktop environment.

The setup a greenfield resource domain and forest trust from an existing tenant with a two way trust. Basically everything was correct but the logon from the users would always get terminated at the desktop with invalid credentials.

After a short discussion and remote session and the error messages in the logs with an invalid CRL it was clear that was the issue. Troubleshooted the AIA/CRL locations and basically the defaults where still in play, explained that default push in AD isn’t a recommended approach. If any client can’t access the CRL it will give a deny on further actions (and other clients that don’t understand AD or are joined to AD won’t work as well).

Below screenshots depict a default ADCS installation which in turn pushes out the default legacy templates and also the CRL to LDAP which I see much too often at customers:

Resolution for the CRL error was to revoke all the certificates for usage with FAS, change the CRL/AIA location to a routable and reachable HTTP listener instead of LDAP (preferably an HA setup with a load-balancer in front of it) and push out the new CRL. Afterwards logons where using the SSO capabilities.

Hope it helps!

Notes from the field: VMware UAG reverse proxy why doesn’t it work!

When configuring VMware UAG as an reverse proxy I’ve encountered some issues last year that as far as I could see wasn’t all to well documented. My reference article for the configuration was the following: https://techzone.vmware.com/configuring-web-reverse-proxy-identity-bridging-vmware-unified-access-gateway-vmware-workspace-one-operational-tutorial#985671

Basically when you follow it to the letter in your test deployment and with a test site you will not have a working reverse proxy URL. At the time when I encountered this I’ve logged a GSS support case and in the troubleshooting process it was clear that the proxy pattern set wasn’t working whatsoever, the correct one should be (|/(.*)|) instead of (|/intranet(.*)|)

My understanding was that if you would configure the instance id and configured the proxy pattern accordingly it would work but that wasn’t the case. Only when not referencing it and just passing it through it began to work.

When configuring multiple reverse proxy URL’s be sure to create corresponding proxy host patterns on the instance id’s

See the following screenshots for a working reference when using UAG as a reverse proxy for Exchange 2019 and Citrix StoreFront 1912

Hope it helps!

Notes from the field: VMware Horizon Enrollment Server and Core O/S

Recently had an deployment with a customer who has a mandate core o/s deployments are preferred unless the product doesn’t support a core o/s installation.

Well for this deployment we created two core o/s subordinate ADCS servers with the enrollment server software installed and configured. Everything is working fine and dandy, no issues and seems like its golden for production deployment.

Guess again.. all the Horizon products aren’t supported for a core o/s installation, yes it will work most of the time but if you’ll find any errors or in need of GSS support then most likely you would need to install a GUI variant. I’ve had two support cases open and both of them state it’s not supported even though the documentation isn’t explicitly stating that.

Take a look at https://docs.vmware.com/en/VMware-Horizon-7/7.10/horizon-installation/GUID-30AA88CF-8CDF-42E5-97D4-D75B2171434B.html for the ESB release and https://kb.vmware.com/s/article/78652 for the Horizon 8 2006 release.

I’ve asked for a documentation update in the support case and also left feedback that the articles need updating. Please do the same so that we can prevent others from installing a unsupported setup.

Hope it helps!

Notes from the field: VMware Access connector support LDAP Signing and Channel Binding

Quite recently I’ve encountered a random synchronization error that VMware Access connector could not synchronize and would error out with the following error: “Connector communication failed because of invalid data: The specified Bind DN and password could not be used to successfully authenticate against the directory”

At first I stumbled upon the known issues list: https://docs.vmware.com/en/VMware-Workspace-ONE-Access/19.03/rn/VMware-Identity-Manager-1903-Release-Notes.html#knownissues and checked if the computer name was the same as the name in the domain field and that was all correct.

Eventually it came to light that the LDAP Signing and Channel Binding hardening were implemented according to the latest Microsoft update. Well then you can also get this sort of behavior. The solution is present in an hotfix for the connector software.

Knowledge base article can be found here: https://kb.vmware.com/s/article/77158 and the hotfix can be found after logging in at my vmware and the components of Access/Identity Manager

Notes from the field: Citrix XenMobile / Endpoint Management Per App VPN not working for iOS

This was quite a nice one to troubleshoot, turns out there is a new configuration point for per app VPN and iOS devices, at least it was for me.

If you follow the configuration at https://www.citrix.com/blogs/2016/04/19/per-app-vpn-with-xenmobile-and-citrix-vpn/#:~:text=With%20the%20iOS%20per%20app,applications%20installed%20on%20the%20device. you’ll end up with a config that won’t open up a VPN when accessing the browser. Solution for this is to change the default provider type in the policy from App Proxy to Packet tunnel also mentioned here https://docs.citrix.com/en-us/xenmobile/server/policies/vpn-policy.html and explained it means the following:

Provider Type: A provider type indicates whether the provider is a VPN service or proxy service. For VPN service, choose Packet tunnel. For proxy service, choose App proxy.

Hope it helps!

Notes from the lab: Windows firewall profile not correct after reboot

Just thought of leaving a quick win here. Did you ever had the firewall profile of Windows not correctly mapped after reboots etc.?

This is because after a reboot the Domain Controllers put it in e.g. public profile and this will get passed on to other servers as well. This will effect in not being able to manage machines because of firewall blocks etc.

Solution is to restart the “Network Location Awareness” service and dependent “Network List Service”.
This will reset it to domain profile and after reboots of the other machines which have this it will be updated to domain profile as well. Or restart the service as above that will also do the trick.

Hope it helps!

Notes from the lab: Citrix ADC IP Reputation

I’ve been playing around with the Citrix ADC IP Reputation feature – https://docs.citrix.com/en-us/citrix-adc/13/reputation/ip-reputation.html in the lab for some time and to be honest it’s such a small but very effective feature which I almost never see active, why is that?

If you’ve gotten a premium licensed ADC appliance it’s a simple right click>enable and you put in the necessary arguments in a responder policy. See the following article for a quick how to video – https://www.youtube.com/watch?v=WedxwiEVuG4 and basically that is it. The requests are going to be filtered on a Webroot service provider for malicious IP database and you can then drop those from ever getting at you network. (and put in a nifty log action so that you can filter as a syslog entry in Citrix ADM

I put a global responder in place with the expression: CLIENT.IP.SRC.IPREP_IS_MALICIOUS and a reset with accompanied log entry: CLIENT.IP.SRC  + ” connection was dropped by Responder Action for malicious IP when accessing ” + HTTP.REQ.URL the results were pretty much mind blowing, see the following screenshot:Since the exploit CVE-2019-19781 – Vulnerability in Citrix Application Delivery Controller, Citrix Gateway, and Citrix SD-WAN WANOP appliance – https://support.citrix.com/article/CTX267027 the focus was pretty much around that but with the above rule in place I got more hits on the reputation feature than on the mitigation responder in that time. (And these are probes/attacks we don’t know about!)

So to sum it up this is a very good starting point if something like Application Firewall is a bridge to far but most definitely will improve your security with a simple setup.

Hope it helps.