Tag: network

Building the lab 3: NSX

On By sam.aaronIn Daily, Home Lab, NSX, Projects, Tips, vCloud, VMwareLeave a comment

Now that vCenter is installed and configured, I am ready to move onto the installation of NSX. NSX for vCloud Director (vCD) is a tad simpler installation than implementing a standard NSX deployment for vSphere. Luckily, the good folks over at SysadminTutorials have a most excellent walkthrough on NSX for vCloud Director. Networking is my Achilles heel, so I struggle with it. When I write about networking, I will try and detail out areas that are confusing to me.

For my environment, I have installed the following components:

  • vSphere 6.5 (vCenter & ESX 6.5)
  • NSX 6.3.5


TIP JAR

I followed the Sysadmin Tutorial to perform the NSX installation in my lab. This tutorial was spot on (even for version 6.3.5), however, there are some things to note regarding the installation for my environment.
Placement: Remember, in my environment the vCenter manages the compute cluster. The NSX Manager will be installed on the management host next to the vCenter server. When I deploy the NSX controllers, each controller will be installed in the compute cluster — not the management host (as the tutorial suggests).
NSX Controller IP Pool: For me, I consider the NSX Controllers an extension of the management plane. I also realized that I would only be installing two controllers. This goes against best practice and the recommended ‘three’ from VMware. Therefore, the IP Pool I created for my controllers was a pool of two IP addresses. During the install, I assigned a controller to each host within the compute cluster.
VXLAN IP Pool: When configuring VXLAN (steps 32-36), I again only created a pool of two IP Addresses for each of my ESX hosts within the compute cluster. Since these are VMKernel NICs on the ESX hosts, I kept them on the management network.
MTU Size: I cannot stress enough how important this is. If you can create Jumbo Frames throughout the environment, you will be saving yourself from heartache. The MTU setting that is absolutely required for NSX is 1600. But if you are going to implement jumbo frames, go all the way and give it 9000.


In my experience, I’ve seen this be the issue that killed connectivity and created fragmentation where it didn’t need to be, among other things. On one of my previous engagements, the customer utilized an encrypted Active Directory. During a domain join, I would have machines throw errors. When we troubleshot, what we found was that the encrypted traffic could not be fragmented. The packet size was 1538, MTU was 1500 on their network. This authentication packet was tossed out every single time preventing the machines from joining the domain. This is just one example where this has shown its’ ugly face. My recommendation: check from end-to-end that your MTU is set appropriately.





After the installation of NSX, this is what my environment looks like. The green is to indicate that the vCenter is managing the Compute Resources. As you can see, it is a simple installation so far.

Up next, I will build a CentOS machine and install the vCloud Director Cell.

Prevent the incrementing of eth devices on Linux systems after guest customization of a cloned VM

On By sam.aaronIn HowTo, LinuxLeave a comment

I’ve ran into this issue before and found the following article written by Chris Greene from Orchestration.io.

After the guest customization process runs on cloned VMs in some VMware products, you may notice that on your Linux systems the eth device number gets incremented. For example, when the system is first built, the first eth device will be eth0. If the system is cloned & customized, the eth device will become eth1. This may not be a problem on some systems, but people often need/prefer the first eth device to be eth0 or at least to not change after the system is customized.

The issue arises because of old entries in the udev network file – /etc/udev/rules.d/70-persistent-net.rules. After an initial install of a Linux system that has a NIC with a MAC of “00:50:56:02:00:7c”, /etc/udev/rules.d/70-persistent-net.rules will look something like

When you perform a clone & customization (as in creating a new vApp from a template in vCloud), the source VM is cloned and has NIC with a new MAC address. When the cloned VMs boots, udev notices the new NIC and updates /etc/udev/rules.d/70-persistent-net.rules with the new NIC and gives it the name eth1.

A new file named /etc/sysconfig/network-scripts/ifcfg-eth1 will be created that points to the eth1 device

Now when ifconfig is ran, you will see eth1 instead of eth0.

To prevent the issue from occurring, delete the /etc/udev/rules.d/70-persistent-net.rules file before shutting down the VM and turning it into a template. This will cause a new /etc/udev/rules.d/70-persistent-net.rules to be created when the customizing VM boots up. The new file will only contain the NICs on the system and they should be labelled as eth0, eth1, etc.

Another thing you may want do before shutting the VM down to be added as a template is modify /etc/sysconfig/network-scripts/ifcfg-eth0 so that ONBOOT is set to no (ONBOOT=no). I’ve seen issues in vCloud where multiple vApp templates are being deploying onto the same network and the VMs have the same IP (that was initially on the VM before it was turned into a template). Then the systems boot, ifup is ran, which runs arping. I’ve seen arpping return an error in these situations, which causes VMware tools to not start. This then causes guest customization to fail since VMware tools is relied up by guest customization.

Disable Flow Control

On By sam.aaronIn Daily, ESX, HowTo, VMwareLeave a comment

HP C7000 Blade Chassis
HP C7000 Blade Chassis
I was onsite this past weekend doing a vSphere 5.1 turnkey install. The turnkey consisted of a C7000 Blade Chassis, and two ESXi 5.1U3 hosts. With this new installation, the customer was adding to a new 10Gbe environment to his network. There were also a few settings that needed to be made, specifically, disabling Flow Control on the ESXi Hosts for Mgmt, Data, and vMotion.

The Flex10 Virtual Connect Modules in the chassis do not have a method of disabling flow control, each ESXi Host would auto-negotiate with Flow Control On for both transmit and receive. Personally, I’ve never had to worry about Flow Control within the environments that I’ve set up. But on this engagement, I was working side by side with a guy who lives, breathes, and retransmits network. Trust me, he knows his stuff. It was his recommendation to disable Flow Control. Realizing I had to figure this out quickly, I turned to the number one tool in my toolbag – Professor Google.

Flexing my “Google-Fu”, I found my answer right away: VMware’s KB Article 1013413. I opened an SSH session, and ran this command

# ethtool –pause tx off rx off

That’s Great! One problem. When I reboot, Flow Control is going to reenable because of the auto-negotiate. So how do I make this persistent/permanent? Another well versed Google search brought me to VMware KB Article 2043564. Once this change was applied, reboots did not reenable Flow Control. My network guy was happy, and hey – I learned something new.



So the next question then, is how do I add this change into a kickstart file. Hmm….. Stay tuned folks. I’ll see if there’s a way to do that.

Project: HomeLab

On By sam.aaronIn Daily, Home Lab, ProjectsLeave a comment

Part 2: Network Planogram

Up until now, my home network has been pretty simple and fairly common-place. It consists of the following: Internet > Cable Modem > Linksys Wireless Router > LAN/WLAN. The service has been consistent over the years, but for this HomeLab Project – it too needs an overhaul. The new network needs to be seperated between the home LAN, and the LAB. The two subnets will also need to be trusted so that machines can communicate between the two networks.

My Linksys Router (a WRT54GS) has been running dd-WRT on it for a few years now. Unfortunately, the firmware version (v6) of the Linksys will only allow me to run the “micro” version of dd-WRT. This prevents me from being able to set up VLANs and split the home LAN from the Lab LAN.

I plan to implement a Cisco 2600 Router that has been lying around collecting dust into this new design. This should allow me to split the two networks using VLANs.


Since I have very little background in configuring and setting up routers, this implementation will be a good challenge for me. However, I will need help (Google can only do so much). I have contacted a few of my Cisco friends to bounce ideas and designs off them. With their help, I have a rough outline and a generic/basic router config to get me started.