Unhandled Exception when logging into ESXi Host client

Unhandled Execption Error
Ran into this weird error after powering up the C.R.I.B. and logging into the ESXi Host to boot up VMs. This error popped up while using Safari, Chrome, and Firefox.

Unhandled Exception (1)
Cause: Error: [$rootScope:inprog] http://errors.angularjs.org/1.3.2/$rootScope/inprog?p0=%24digest

The error presents itself with two buttons: Reload and Details. The picture displayed shows the details of the error that I got. You can hit reload, attempt to log back in, and rinse and repeat the process. Or you can select Details, where you can only ‘close’ the error and repeat the process.

A quick check with Professor Google finds this is an issue mentioned in the VMware Communities. Luckily, the entry had an answer attached. (Thanks, “rshell”). It’s not a fix, but it is an answer.

I can’t explain why the error occurs, but I can explain what causes it to pop up. This error occurs when you land on the logon page, enter credentials, and then hit on the keyboard instead of clicking on the button. If you click the login button, you have no errors and vSphere loads normally.

C.R.I.B. The Logical Layout of My HomeLab

C.R.I.B. – Stands for Computer Room In a Box. This is the name I have given my homelab. I’ve used my C.R.I.B. to educate myself, experiment with things, and demo products to my customers.

As you’ve read in previous posts, my homelab has evolved over the years. Currently, I run one physical Supermicro Super Server attached to a Synology DS 1512+ Array, connected to a gigabit switch. Many of my friends and co-workers have asked how I run everything on one physical box — which I am going to call pESX. I’ve tried to explain it and draw it out on a whiteboard. However, until you’ve seen it drawn out, the explanation gets confusing — unless you’re familiar with nested virtualization.

I utilize nested virtualization to expand the capabilities of the C.R.I.B. without additional physical hardware. If you are unfamiliar with nested virtualization, it is the ability to run ESX as a virtual machine — which I will call vESX. (William Lam has written numerous articles on how to do it. Just google “William Lam” and “nested virtualization” if you want more info.) The entire CRIB is accessible from my home network — which is a lifesaver, as I do not have to work in my office. I can access it via VPN or the couch.

CRIB
Computer Room In a Box
A pfSense virtual machine (GATEWAY) was implemented to act as firewall and router to the entire virtualized environment, including the nested layer. The pESX is on the base network with a vmkernel attached to the pfSense Virtual Machine to allow for manipulation and modification of the firewall rules and network configuration. All traffic in and out of the virtual environment will pass through the pfSense VM. The firewall in the pfSense provides isolation as well as communication between the various networks.

All of the infrastructure virtual machines sit on the first virtualization layer – this is considered as the “Management Cluster”. However, this cluster, is only made up of the one physical ESX Host (pESX). Normally, we would want multiple hosts for HA redundancy. (But this is a lab, and I’m on a budget.) The vESX Virtual machines sit in this layer as well. The vESX VMs have a direct connection to the base network for access to the iSCSI storage Array. vESX make up the ESX Compute Resources of the “Payload Cluster”. These clusters; the Management Cluster and the Payload Cluster, are a VMware architectural design best practice. The infrastructure is made up of your basic VMs; a Domain Controller (DC), a SQL Server (SQL), the Management vCenter (vCSA6), a Log Insight Server (LOG), and a VMware Data Protection VM (VDP) for backups. In addition to these VMs, the vRealize Automation (vRA) VMs and Payload vCenter (PAYVC01) also sit in the management cluster. This self-service portal deploys to a vCenter (PAYVC01) endpoint controlling the compute resources of the Payload Cluster.

The Payload Cluster is made of three virtual ESX Hosts (vESX) and provide the various resources; network, CPU, RAM, & Storage, for consumption by vRA or other platform products. There is an Ultimate Deployment Appliance (UDA) VM providing the ability to deploy scripted ESX images. This provides the ability to quickly rebuild the hosts, if needed.

This is just the base. I am in process of deploying NSX into the environment to provide the ability to deploy multi-machine blueprints within vRA. In addition, I intend on exploring SRM integration with vRA.

How to upgrade a simple vRA 7.0 instance to vRA 7.0.1

Just this week, VMware released vRealize Automation 7.0.1 (vRA). It contains many bug fixes and some enhancements to the vRA platform. I was excited for it to come out and was anxious to perform an upgrade in my home lab.

I will advise caution and planning in any upgrade of your environment. But I would stress heavily on the planning. You should know your dependencies before you attempt an upgrade, and always. ALWAYS, read the Release notes before you start the upgrade process.

The following process is for a simple vRA instance. This is the Proof Of Concept build, sometimes referred to as a “Lab” or “Sandbox” build. However, these steps can be modified for a fully distributed vRA instance.

Here is how I upgraded my lab.

1) Take snapshots of the vRA Cafe Appliance, IaaS VM, and SQL VM.

2) Shutdown the vRA Services
     SSH into the vRA Cafe Appliance and shutdown the vco-server, vcac-server, apache2, and the rabbitmq-server services.

  1. Run the below commands to stop the above listed services:
    • #service vcac-server stop
    • #service apache2 stop
    • #service rabbitmq-server stop
    • #service vco-server stop


    You can check that the services have stopped using the status syntax: #service vco-server status

  2. Log into the IaaS Virtual Machine and stop the below listed vRA services.
    • All VMware vCloud Automation Center agents
    • All VMware DEM Workers
    • VMware DEM Orchestrator
    • VMware vCloud Automation Center Manager Service


3) Download the vRealize Automation Appliance 7.0.1 Update Repository Archive ISO.

4)Upload the ISO to a datastore, and mount the iso to the vRA Cafe Appliance’s CDRom.

VM Settings
5) Open a browser and log into the vRA Cafe. Then Navigate to the “Update Tab” –>> “Settings”.

6) Change the Update Repository to “Use CDRom Updates”. Click on “Save Settings”.

Use CDRom Updates
Use CDRom Updates

7) Select the “Status Tab”.

8) Click on “Check For Updates”.

Check For Updates
Check For Updates

9) An update should be found (as shown in the photo above). Click on “Install Updates”.

10) Wait for the update to complete. This took approx 30 minutes for my lab.

Install Updates
Install Updates

11) Once the updates complete, you will be notified to reboot the vRA Cafe Appliance.
Reboot Notice
Reboot Notice

12) Once the vRA Cafe Appliance has completed the reboot, log back into the vRA VAMI and verify the version.
Updated Version
Updated Version

This completes the vRA Cafe Appliance upgrade. Now it is time to focus on the IaaS Server.

13) Open a console or RDP session into the IaaS Server and log into the machine with the vRA Administrator Service Account.

14) Open a web browser and browse to the vRA Cafe Installer page. “https://[vRA Appliance FQDN]:5480/installer”

15) Download the “DBUPGRADE SCRIPTS”.

16) Verify the Java Path in the Environmental variables.

Java Path
Java Path

17) Open the File Explorer and browse to the folder where you downloaded the “DBUPGRADE.zip” scripts file. Extract the DBUpgrade.zip file.

18) Open an elevated Command Prompt.

19) Change the directory to the location of the DBUpgrade Extraction Folder.

20) NOTE: Verify that the vRA Administrator Service Account has the SQL sysadmin role enabled.

21) Run the following command to update the SQL Database:
      # dbupgrade -S sql.dwarf.lab -d vra -E -upgrade

Replace sql.dwarf.lab with the FQDN of your SQL server.

DBUpgrade Script
DBUpgrade Script

The process may take a few minutes to complete.

22) Return to the vRA Cafe Installer page. “https://[vRA Appliance FQDN]:5480/installer”. Download “IaaS_Setup”.

23) Browse to the downloaded file in File Explorer. Right-Click the file, and “Run as Administrator”.

vRA 7.0.1 IaaS Installation - 1
vRA 7.0.1 IaaS Installation – 1

vRA 7.0.1 IaaS Installation – 2
vRA 7.0.1 IaaS Installation – 2

vRA 7.0.1 IaaS Installation – 3
vRA 7.0.1 IaaS Installation – 3

24) Select “Upgrade”
vRA 7.0.1 IaaS Installation – 4
vRA 7.0.1 IaaS Installation – 4

vRA 7.0.1 IaaS Installation – 5
vRA 7.0.1 IaaS Installation – 5

25) Fill in the Blanks.
vRA 7.0.1 IaaS Installation – 6
vRA 7.0.1 IaaS Installation – 6

NOTE: For the SQL Connection. If you are not using SSL, uncheck the option to “Use SSL for Database Connection”; else you will experience the following error.
vRA 7.0.1 IaaS Installation – 7
vRA 7.0.1 IaaS Installation – 7

26) For my lab, I had to remove the SSL connection between the IaaS Server and the SQL Database Server.
vRA 7.0.1 IaaS Installation – 8
vRA 7.0.1 IaaS Installation – 8

vRA 7.0.1 IaaS Installation – 9
vRA 7.0.1 IaaS Installation – 9

vRA 7.0.1 IaaS Installation – 10
vRA 7.0.1 IaaS Installation – 10

vRA 7.0.1 IaaS Installation – 11
vRA 7.0.1 IaaS Installation – 11

27) The upgrade installation will take some time to complete. I recommend going and grabbing a drink. The process took approx 30 mins for me for it to complete.
vRA 7.0.1 IaaS Installation – 12
vRA 7.0.1 IaaS Installation – 12

28) The upgrade finishes.
vRA 7.0.1 IaaS Installation – 13
vRA 7.0.1 IaaS Installation – 13

29) Click finish and reboot the IaaS Server.

30) When the server comes back online. Log back in and verify that all vRA services have restarted.

vRA Services
vRA Services

31) Log back into the vRA Cafe Appliance and check all Services are returned to “Registered”.
Cafe Services
Cafe Services

If everything happened without any issues, then you have successfully upgraded vRA from 7.0 to 7.0.1. Go log into your portal and check it out!

vRA Portal Login
vRA Portal Login

How to Mount a USB Drive as an ESXi Datastore

Seagate Backup Plus Portable Drive
Seagate Backup Plus Portable Drive

Recently, I have upgraded my homelab – yet again. There will be an oncoming post about the hardware I chose and how it is set up. This time, my lab is mobile. I can take it with me on the road or leave it at home.
One of the features that I wanted to be able to capture and utilize was VMware Data Protection (vDP) to backup my Infrastructure and Important VMs. However, there was a small hurdle that I needed to overcome – storage – and how do I make it mobile. VDP requires – at minimum, a 2TB datastore for backup storage. Normally, you would utilize your storage array and carve out some space for the backups; I wanted to see if I could do something different – something radical. I wanted to use my USB3 USB drive.
Continue reading “How to Mount a USB Drive as an ESXi Datastore”

ESXi BIOS Power Settings Best Practice

Dell Poweredge R720 Server BIOS Power Settings
Dell Poweredge R720 Server BIOS Power Settings

There is a movement to change the best practice regarding the bios power settings on an ESXi host. In the past, you would set the power settings in the BIOS to max and be done with it. You did this because previous versions of vSphere did not work well with the various C-States of the processor (ie., When the workload on the host would drop, the CPU would drop to a lower power setting. When it would go to wake and draw the CPU back to full power, the host would crap all over itself.).

vSphere 5.5 introduced a new feature that allows VMware to better utilize the C-States of some of the newer processors. Thus allowing the CPUs to change power and speed states without affecting the performance or behavior of the Host. (YEAH!!)

I know we are happy about this, but not so fast…

This link describes the behavior and a couple of performance benefits from the change — however, it should also be noted, that this change is dependent on the type of workloads in your environment. If you have a heavy IO Intense workload or a time sensitive workload, you may still want to have that host on max power, high performance, etc.

http://blogs.vmware.com/vsphere/2014/01/bios-power-policies-affect-performance.html