Notes from the lab: Citrix ADC and VMware ESX 7u1

First things first. Citrix ADC at this time isn’t supporting VMware ESX 7.0.1 according to the following article: Support matrix and usage guidelines (citrix.com)

This is something that obviously will get supported in due time. But for the people who are running it just as I am in the lab you would see issues like the ADC instances would lose connectivity or will not load the appropriate network drivers at boot. This is because of the VMXNET3 interface which is causing issues.

Temporary workarounds include just reboot the dam thing until it works… or flip the network interface to E1000 and you have a solid booting/working appliance, well not entirely because the GUI seems to slow down over time to almost not working. I decided to flip it back to VMXNET3 and just do the reboot loop until it works.

Keep in mind that you need to copy/paste your MAC-address before removing the interface and readding them to keep your license etc. nice and working.

Hope it helps!

Notes from the field: VMware Horizon sessions disconnecting after syslog changes on UAG

On a recent project where we have VMware Horizon 7.13 and UAG 20.09 appliances for the external connections some strange behavior was observed when putting in the syslog URL entries.

After adding or removing entries here and saving the settings all the connections through the UAG will get terminated. Finding this behavior strange as to you don’t do anything that special so why disconnect these sessions.

Had some discussion on the vExpert slack channel and quickly came to light that it looks like an regression issue.

The following article describes the issue User sessions are disconnected through VMware Unified Access Gateway when the configuration settings are changed with the UAG Admin UI (56487) and this should be resolved in later versions.

The version tested here is the release that accompanies Horizon 7.13 build and that is UAG 20.09.

The versions tested by Jesper Alberts were Horizon 2012 with UAG 20.09 and the same problem.

The versions tested by Stephen Jesse were Horizon 7.10 with UAG 3.7 and 3.10 without any issues.

So to sum it up if you have the builds in question it might be a good action to log a support case.

Hope it helps!

Notes from the lab: Bye Bye VMware View Composer

I was upgrading my lab to VMware Horizon 2012 and yes shame on me I still had an composer in my setup. It was already mentioned that VMware Composer is deprecated from the 2006 release but now in 2012 it will block your upgrade when you still have it enabled. Only after disabling composer on your vcenter the upgrade will succeed and afterwards composer will be gone as an configuration item.

See the following screenshots for detailed information:

Notes from the field: Citrix StoreFront forcing connections through Citrix Gateway

On a recent customer project there was the need to migrate off of VDA TLS encryption and migrate the connections from StoreFront to Citrix Gateway.

The customer previously had StoreFront direct connections and used the VDA TLS encryption setup to provide a TLS encrypted session to the desktop or applications.

The VDA TLS encryption setup was too much engineering labor for the day 2 day operations and therefore they asked for a alternate solution but still provide the client>desktop as an TLS encrypted session.

Here we have two options, the first is to use Citrix Gateway and StoreFront as authentication but this introduces the users with a new logon screen and then delegates the credentials with json to StoreFront.

The second is forcing the connections from StoreFront through the Citrix Gateway by the means of optimal gateway routing, and we don’t have any user experience changes because the logon point is still StoreFront.

Option two was chosen and after a quick and simple deployment a seamless migration with optimal gateway routing is in place.

First all the preparation in place is creating a new Citrix Gateway DNS record and a new StoreFront load-balancer IP into the current setup will be migrated, afterwards configure the CVAD wizard on the NetScaler for a simple Citrix Gateway deployment and unbind any authentication policies because these will not be used. Afterwards configure all the necessary settings for a standard Citrix Gateway deployment and propagate these changes across the cluster. When all this is done edit the web.config file of the store that got configured under the primary StoreFront servers IIS inetpub directory and search for: optimalGatewayForFarmsCollection and make sure there is an entry with optimalGatewayForFarms enabledOnDirectAccess=”true” and save the file. Propagate the changes and after that migrate the old DNS entry to the new StoreFront ip. You will see that after logon the desktop brokering is force through Citrix Gateway.

The following reference articles where used for configuration and testing:

How to Force Connections Through NetScaler Gateway Using Optimal Gateway Feature of StoreFront (citrix.com)

How to Configure Authentication at StoreFront using NetScaler Gateway – NetScaler Configuration (citrix.com)

FAQ: Configuring Authentication at StoreFront using NetScaler Gateway (citrix.com)

SSL configuration on VDA (citrix.com)

Notes from the field: VMware Horizon Instant Clone and Imprivata OneSign

On a recent project consisting of an VMware Horizon instant clone setup and Imprivata OneSign in the desktop for SSO capabilities I’ve encountered some strange timing issues.

Normal logins through the horizon client via connection server would be ok with the OneSign agent online, logins through the UAG without TrueSSO would also be okay. (so it seemed)

TrueSSO enabled because the total solution is a Workspace ONE deployment and we want to use one login of course regarding credentials introduced a problem. The login process would work just fine but the agent of Imprivata would stay offline only to be online after a reconnect to the desktop.

A long troubleshooting process started with the team and also a support case opened with Imprivata pointed us in the direction that there was an issue all along with the base setup as well.

Logging in to the VMware console of an desktop without the UAG or TrueSSO in place would provide the same result, an offline OneSign agent and the second logon or disconnect/reconnect even in the console would resolve it.

Long story short after that finding we did an test to restart the OneSign agent before logon of any user (tested this through Ivanti AM) and then let the user logon and presto no more issue.

We then created a script to be used in ClonePrep so that we don’t have to rely on Ivanti AM and after that issue is also resolved.

But now comes the kicker. The solution at the time was an Horizon 7.10.x ESB release and for customer requirements and lifecycle management to be future ready we needed to go to the 7.13 release of Horizon (not perfectly documented but this is the new ESB release as well and last significant release of Horizon 7)

After this update to the connection servers the ClonePrep scripts would time-out and not function correctly anymore. Removing that from the pool did blow my mind because after that the complete issue was gone and the OneSign agent would be online all the time no matter what.

Tested this with an old image and the 7.10.x release of Horizon installed with the 7.13 connection server as backend and indeed no more issues and everything fine. The root cause seems to be “something” in instant clones that got fixed/changed in 7.13

To summarize, the fix for an 7.10.x release is to use any means of restarting the OneSign service before any user will logon and for the 7.13 release it’s no issue at all. (I’m betting this will also be the case for Horizon 8 but haven’t tested that with Imprivata)

Some reference articles:
The Imprivata URL’s you will need an active account at Imprivata to view them

How to enable Imprivata Agent trace

Does Imprivata support VmWare Instant Clone for VM Desktops?

Configuring Kerberos Authentication (imprivata.com)

Configuring Support for VMware True SSO (imprivata.com)

Information on Horizon 7 Extended Service Branch (ESB) (52845) (vmware.com)

VMware Horizon 7 version 7.13 Support Plan (81189)

ClonePrep Guest Customization (vmware.com)

Notes from the field: Citrix FAS request not supported

On a recent Citrix FAS deployment I’ve encountered the following error: “Request not supported” when logging in to a published application or desktop.
Article https://support.citrix.com/article/CTX218941 explains that re-enrollment of the domain controller authentication template or another custom template for Kerberos usage should resolve the error.

A little bit of a background on the environment, an already working Microsoft ADCS environment was in play and in use for other services. From a design/security perspective it was designed that two dedicated Microsoft ADCS servers would be used and two Citrix FAS servers connecting these new servers. The setup was working as expected but only above error would keep coming when trying to access an application or desktop.

We tried re-enrolling the domain controller authentication certificate and this didn’t do the trick, then we decided to let the Domain Controllers get the certificate from the new dedicated Microsoft ADCS servers for Citrix FAS and this did do the trick but with a side effect the chain is changed and other services would be negatively impacted so a rollback was needed.

With this information a Microsoft support case was created and ultimately they confirmed that what is mentioned in the Citrix support article should do the trick. Ok we got confirmation and yes it indeed does work when using the new ADCS servers but the issue of the original ADCS environment was still a mystery.

So next up we decided to repoint the Citrix FAS servers to the existing Microsoft ADCS server to root out any chain or other issues that might be in play. The result was exactly the same and a not supported request as the end result.

Digging deeper in the Microsoft ADCS environment it was after checking the “NTAuthCertificates” store that the existing server wasn’t there and the new servers were. This explained the smartcard logon not working when using the existing environment because an requirement for smartcard logon is that the “NTAuthCertificates” store has the issuing certificate authority propagated. After adding the certificate and waiting for replication and a reboot everything was working as expected, also when moving to the new Microsoft ADCS environment for certificate issuing.

See the following screenshot of the Enterprise PKI snap in MMC in which you can check and/or add the missing certificate:

See the following articles for extra information:
https://docs.microsoft.com/en-us/troubleshoot/windows-server/windows-security/import-third-party-ca-to-enterprise-ntauth-store
https://support.microsoft.com/en-za/help/281245/guidelines-for-enabling-smart-card-logon-with-third-party-certificatio

Notes from the lab: Microsoft ADFS and VMware UAG

You don’t see many configuration articles around ADFS and UAG and that’s why I would like to share my setup.

First things first, I’m expecting that there is an working Horizon environment with True SSO enabled for access to the desktop. And a working ADFS environment to add a new application to test with.

My setup:
1 x ADFS for internal usage
1 x WAP for external usage
1 x UAG v3.10 – dedicated for ADFS with its own URL
1 x UAG v3.10 – dedicated for WS1 with its own URL
2 x Horizon Connection Servers
2 x Horizon Enrollment Servers

The following articles helped me in setting this up:
https://docs.vmware.com/en/Unified-Access-Gateway/3.10/com.vmware.uag-310-deploy-config.doc/GUID-E4C8B88F-C771-4829-ABBE-12F7FBF517C3.html
https://communities.vmware.com/thread/625006
https://thevirtualhorizon.com/2019/12/14/integrating-microsoft-azure-mfa-with-vmware-unified-access-gateway-3-8/
https://techzone.vmware.com/enabling-saml-20-authentication-horizon-unified-access-gateway-and-okta-vmware-horizon-operational

Setting this up starts with downloading the “/FederationMetadata/2007-06/FederationMetadata.xml” file of your ADFS setup and saving this on your configuration machine

Now go to the UAG admin appliance on the management port https://xxx:9443 and scroll down to the Identity Bridging Settings, select the gear icon of “Upload Identity Provider Metadata” and in the next screen press on the select link where you can upload the metadata file of ADFS.
Now go to the Horizon settings on the top and select the gearbox and in the next screen select more to show the auth methods to be used. For True SSO select SAML. Next select the uploaded Identity Provider which is now visible and click on the “Download SAML service provider metadata”. Save everything and close the admin portal.

Next in ADFS configure a new “Relying Party Trust” claims aware application and import the downloaded SAML service provider metadata file from UAG
When completed edit the “Claim issuance policy” for the application and create a rule with attribute store active directory selected and provide the input for “User-Principal-Name” with “Name ID” as outgoing claim type.
Edit the application and select encryption and click on remove the encryption certificate. This enabled will not give an valid SAML assertion or logon.

This is all that is needed in ADFS, the application can be assigned anyway you want. The last steps are in Horizon itself.

Go to the admin panel of the connection server and configure an SAML 2.0 Authenticator, create one and name it accordingly (don’t forget to enable True SSO on this connector) and make sure it is an static type. Copy paste the federationmetadata.xml content into the SAML Metadata screen and click OK.

Set the delegation of authentication to VMware Horizon (SAML 2.0 Authenticator) to optional or required. The latter prohibits passthrough authentication straight from the servers itself and will always require a SAML assertion.

This is it, afterwards you should see an ADFS logon possible with SSO in place.

Some extra information for any use case with ADFS/Workspace ONE combined then this procedure will not help. For Workspace ONE integration the only working and supported way is to configure VMware Horizon with VMware Access. I’ve tried it configured with VMware Access and the same UAG and you will get an access denied because the SAML configuration is in place at the Horizon Connection Servers instead of the UAG. ADFS can also be integrated with VMware Access and the SSO can be achieved in that way which is a route you would take when using Workspace ONE.

Hope it helps!

Notes from the field: Citrix FAS SSO not working with invalid CRL

Recently I got contacted by a customer who had problems performing an SSO to a newly build desktop environment.

The setup a greenfield resource domain and forest trust from an existing tenant with a two way trust. Basically everything was correct but the logon from the users would always get terminated at the desktop with invalid credentials.

After a short discussion and remote session and the error messages in the logs with an invalid CRL it was clear that was the issue. Troubleshooted the AIA/CRL locations and basically the defaults where still in play, explained that default push in AD isn’t a recommended approach. If any client can’t access the CRL it will give a deny on further actions (and other clients that don’t understand AD or are joined to AD won’t work as well).

Below screenshots depict a default ADCS installation which in turn pushes out the default legacy templates and also the CRL to LDAP which I see much too often at customers:

Resolution for the CRL error was to revoke all the certificates for usage with FAS, change the CRL/AIA location to a routable and reachable HTTP listener instead of LDAP (preferably an HA setup with a load-balancer in front of it) and push out the new CRL. Afterwards logons where using the SSO capabilities.

Hope it helps!

Notes from the field: VMware UAG reverse proxy why doesn’t it work!

When configuring VMware UAG as an reverse proxy I’ve encountered some issues last year that as far as I could see wasn’t all to well documented. My reference article for the configuration was the following: https://techzone.vmware.com/configuring-web-reverse-proxy-identity-bridging-vmware-unified-access-gateway-vmware-workspace-one-operational-tutorial#985671

Basically when you follow it to the letter in your test deployment and with a test site you will not have a working reverse proxy URL. At the time when I encountered this I’ve logged a GSS support case and in the troubleshooting process it was clear that the proxy pattern set wasn’t working whatsoever, the correct one should be (|/(.*)|) instead of (|/intranet(.*)|)

My understanding was that if you would configure the instance id and configured the proxy pattern accordingly it would work but that wasn’t the case. Only when not referencing it and just passing it through it began to work.

When configuring multiple reverse proxy URL’s be sure to create corresponding proxy host patterns on the instance id’s

See the following screenshots for a working reference when using UAG as a reverse proxy for Exchange 2019 and Citrix StoreFront 1912

Hope it helps!