Notes from the field: Configuring OpsGenie (without Atlassian Access) with VMware Workspace ONE Access

OpsGenie can use SAML SSO without the use of Atlassian Access, see the following url: https://docs.opsgenie.com/docs/single-sign-on-with-opsgenie

For the configuration part of Workspace ONE Access just add a new manual SAML 2.0 application and provide the following information according to above article:

  • Single Sign On URL https://app.opsgenie.com/auth/saml?id=”uniquesamlidprovided
  • Recipient URL https://app.opsgenie.com/auth/saml?id=”uniquesamlidprovided
  • Application ID https://app.opsgenie.com/auth/saml?id=”uniqesamlidprovided
  • Username Format = Unspecified

Username Value = ${user.email}

Notes from the field: Configuring Atlassian Access with Workspace ONE Access

Atlassian Access is the SSO portal being used for SSO access across Jira, Confluence etc. for the configuration part see the following url: https://confluence.atlassian.com/cloud/saml-single-sign-on-943953302.html

For the configuration part of Workspace ONE Access just add a new manual SAML 2.0 application and provide the following information according to above article:

  • Single Sign On URL https://auth.atlassian.com/login/callback?connection=saml”uniquesamlidprovided
  • Recipient URL https://auth.atlassian.com/login/callback?connection=saml”uniquesamlidprovided
  • Application ID https://auth.atlassian.com/saml/”uniqesamlidprovided
  • Username Format = Unspecified
  • Username Value = ${user.email}
  • Relay State URL = https://id.atlassian.com/login

Add the custom attribute mappings for firstname, lastname and userprincipalname.

Notes from the field: vCloud usage meter doesn’t meter NSX

A while back I had an support case with VMware support regarding NSX integration and that it wasn’t getting metered by vCloud Usage Meter in a customer deployment. Turns out that Usage meter looks for a Global Transport Zone before the discovery of a Universal Transport Zone and metering can occur. So if you are in a setup that only has Universal Global Transport Zones it is expected behavior to see no NSX monitoring hits being satisfied in Usage meter. This can be resolved by adding a Global Transport Zone as a fictive addition so that it will meter your setup.

Notes from the field: Windows 2019 Storage Replica lock-up on VMware

On one of my latest projects consisting of a new Windows Server 2019 setup on VMware and making use of Storage Replica in a server to server setup for replicating home drives and profiles I came across a random lock-up of the VM and by that inaccessible shares.

The setup was all working until the failover part. It seems there is an delay of some sort and the failover isn’t instant or takes a while to be active with the server being unresponsive and disconnecting any form of management to the VM in question(VM tools are not responding as well and console login will not work in this failover time). I’ve tried the actions again of doing a storage replica failover and I got an BSOD on the VM stating: HAL INITIALIZATION FAILED I’ve tried all of this in a separate test setup and had this working without any problems on Server 2016, and Server 2019. Only this time it gave me this strange behavior. The difference in my own setup is HW level 14 and this new one had HW level 15 and the hosts are 6.7 13981272 build and my own setup is 6.7 14320388 build (older builds have also worked fine for me)

After some troubleshooting and providing the BSOD dump findings to VMware GSS support it became clear that version 10341 of VMware tools was the troublemaker. The solution was to upgrade to the latest 10346 VMware tools. The vmm2core tool provided me with the means of creating a dump file with the VM in question.

Notes from the field: Hyper-V to VMware migrated VM’s cannot install VMware Tools

One of my last projects I needed to convert Hyper-V VM’s to VMware, this all went fine with the offline capability of vcenter converter and the migration succeeded. Only after trying to install the VMware tools this would hang on starting the VGauth services and several other dependencies. For reference the VM’s in question are a mixture of 2008R2 / 2012R2. After some troubleshooting and searching the knowledgebase I stumbled across this article: https://kb.vmware.com/s/article/55798

So for the project I didn’t had any ok to patch the servers that was out of scope for this one, the mitigation was to install older VMware tools (10.2.5 to be exact) afterwards the tools installed fine.

On a side note when finalizing the converted VM don’t forget to delete the hidden older hyper-v network adapter, this can still provide conflicts if not removed.

Notes from the field: vCenter cannot validate SSO domain

Came across a peculiar issue when adding an second vCenter to the same SSO domain and enable ELM.
The first deployment worked like a charm and the second errored out with the following error:

It turns out there is a known bug when using uppercase FQDN in the configuration wizard, the solution is to put it all in lowercase.
see the following link for reference: https://kb.vmware.com/s/article/56812

Notes from the field: UEM/vIDM integration caveats

Not too long ago I encountered some issues when configuring UEM and IDM integration. When providing the vIDM URL in UEM for configuring the integration it would error out with below error:

After some troubleshooting it appeared that the access policies where not properly configured as in the last rule in the default access application ruleset was blocking access. Resolution was editing the default policy and ending it with the password method which is associated with the built-in workspace IDP, after that the integration part is working as expected.

Another configuration task which caught me by surprise was that after the configuration is set up between UEM and vIDM the following errors occurred:

Turned out that the integration between UEM and vIDM is depending on Active Directory integration. The basic system domain accounts (even full admins) won’t work in this scenario. Resolution is configuring an domain account with the necessary admin rights in both tenants and then it will work as expected.

Hope this helps!

Notes from the field: vIDM and o365 modern authentication delay

Just a quick win blog to mention and give a heads-up that when you are in the process of configuring vIDM and o365 you might encounter native clients prompting for authentication and a big ass delay when you flip over the authentication and the requested domain from managed to federated with vIDM. This might be up to eight hours!!! Thanks to the #community #vExpert that I got this answer quite fast because I recalled that Laurens van Duijn put something similar in the vExpert Slack group mentioning that he saw this kind of behavior.

So in summary, do it on a Friday and inform your users.

Big shout out to Laurens van Duijn and be sure to follow him on twitter and his blog

Twitter: @LaurensvanDuijn

Blog: https://vdr.one/

Notes from the field: VMware vCenter /dev/mapper/core_vg-core full

Not too long ago I’ve encountered an vCenter instance blowing up the /dev/mapper/core_vg-core with gigabytes of java dump errors.. Just for reference the customers setup is an dual SDDC with respectively an vCenter at each site comprising of vCenter 6.5 U2 and embedded linked mode enabled.

In troubleshooting mode I’ve encountered the following two articles:

https://kb.vmware.com/s/article/2150731

https://kb.vmware.com/s/article/60161

Afterwards decided to open up a support case. This resulted in a session which stated that they had seen this sort of issues arising in 6.7u1 and higher which root caused against hardware level 13 for the appliance and WIA Active Directory integration.

The customer setup had an hardware level 13 deployment on both sites and only one experiencing the problem, and using Active Directory over LDAP integration.

The resolution of the issue was downgrading the VCSA hardware level to version 10.

Supports way is restoring the VCSA with a VAMI back-up restore, my way was re-register the appliance with the VMX file downgraded to the level needed, see https://communities.vmware.com/thread/517825

Hope this helps!

Notes from the field: VMware vCloud Usage Meter vROps cleanup not working

If you ever are in the proces of cleaning up your vRealize Operations Manager instances and are using vCloud Usage Meter as well you might find yourself in a situation that Usage meter keeps referencing an old node which is deleted.

There is a nice explanatory blog available from VMware to resolve most part of this: https://blogs.vmware.com/vcloud/2018/01/updating-vrops-instance-vcloud-usage-meter.html

But if you find yourself in the situation that the old node is still there in Usage Meter but not referencing an vCenter this won’t help.

Should this happen then we need to do the following on Usage Meter:

  1. Login to the Usage Meter CLI as root
  2. Run sql  to enter the DB 
  3. Run: select * from “VcopsServer”; 
  4. Identify the unwanted vROps node from the table — and note its ‘active’ status and ‘id’ from the associated columns
  5. Run: update “VcopsServer” set “active” = ‘f’ where id = [id]; 
  6. Run the same query from step 3 to verify that the server has been deactivated 
  7. Restart the tomcat services with: service tomcat restart 
  8. Log back into the Usage Meter web-portal 
  9. Delete and reactivate the relevant VC server endpoint to refresh the connection
  10. Force a data collection by changing the minute hand from the ‘Collections’ tab to validate fix

You might need to do a reboot of Usage Meter as well but after that the problem will be resolved.

Hope this helps!