vSphere 6.5 – Converged, Hyperconverged Infrastructure & Virtualization

How to troubleshoot vSAN issues?

January 3, 2022January 3, 2022Pradeep AdapaLeave a comment

Great Comprehensive Post on How to Check for VSAN Issues in your Environment !!

Hope you are doing all great, for today’s post I wanted to put together some of the commands/troubleshooting I’ve had used with VMware vSAN,

Identify a partitioned node from a VSAN Cluster(Hosts)

What a single partitioned node looks like:

~ # esxcli vsan cluster get
Cluster Information
 Enabled: true
 Current Local Time: 2020-10-25T10:35:19Z
 Local Node UUID: 507e7bd5-ad2f-6424-66cb-1cc1de253de4
 Local Node State: MASTER
 Local Node Health State: HEALTHY
 Sub-Cluster Master UUID: 507e7bd5-ad2f-6424-66cb-1cc1de253de4
 Sub-Cluster Backup UUID:
 Sub-Cluster UUID: 52e4fbe6-7fe4-9e44-f9eb-c2fc1da77631
 Sub-Cluster Membership Entry Revision: 7
 Sub-Cluster Member UUIDs: 507e7bd5-ad2f-6424-66cb-1cc1de253de4
 Sub-Cluster Membership UUID: ba45d050-2e84-c490-845f-1cc1de253de4
~ #

What a full 4-node cluster looks like (no partition)

~ # esxcli vsan cluster get Cluster Information Enabled: true Current Local Time: 2020-10-25T10:35:19Z Local Node UUID: 54188e3a-84fd-9a38-23ba-001b21168828 Local Node State: MASTER Local Node Health State: HEALTHY Sub-Cluster Master UUID: 54188e3a-84fd-9a38-23ba-001b21168828 Sub-Cluster Backup UUID: 545ca9af-ff4b-fc84-dcee-001f29595f9f Sub-Cluster UUID: 529ccbe4-81d2-89bc-7a70-a9c69bd23a19 Sub-Cluster Membership Entry Revision: 3 Sub-Cluster Member UUIDs: 54188e3a-84fd-9a38-23ba-001b21168828, 545ca9af-ff4bfc84-dcee-001f29595f9f…

View original post 1,361 more words

Obtain the placement of the physical disk by NAA id on the ESXi Hosts

January 3, 2022January 3, 2022Pradeep AdapaLeave a comment

This is a great Post on How to find the Physical Location of the disks on an esxi host.

VirtuallyVTrue

Here is a simple script to obtain the placement of the physical disk by naa on ESXi hosts

Copy below script and save it on the ESXi host

# Script to obtain the placement of the physical disk by naa on ESXi hosts
# Do not change anything below this line
# --------------------------------------

echo "=============Physical disks placement=============="
echo ""
	
esxcli storage core device list | grep "naa" | awk '{print $1}' | grep "naa" | while read in; do

echo "$in"
esxcli storage core device physical get -d "$in"
sleep 1

echo "===================================================="

done

Run the script:

[root@esxi1:~] sh disk.sh

You will get similar output as per your environment.
Output:

[root@esxi:~] sh disk.sh =============Physical disks placement============== naa.5002538a9823d020 Physical Location: enclosure 1, slot 6 ==================================================== naa.5002538a9823d1c0 Physical Location: enclosure 1, slot 3 ==================================================== naa.58ce38ee204ccd59 Physical Location: enclosure 1, slot 7 ==================================================== naa.5002538a9823d070 Physical Location: enclosure 1, slot 1 ==================================================== naa.5002538a9823d040 Physical…

View original post 33 more words

How to Find the NIC Driver Version on ESXI Host and get the Correct Driver from VMware

December 9, 2021Pradeep AdapaLeave a comment

Recently, I had to Search for an QLogic 2x25GE QL41262HMCU CNA NIC driver to update it on multiple Dell R740XD hosts. It’s been a while since I used the Update Manager (vSphere 6.7 environment) and hence writing this post.

First thing is to SSH into an esxi host and then execute the following commands to check the firmware/driver version of the vmnic you want to update (In my case all my vmnics are Qlogic CNA NIC’s)

esxcli network nic get -n vmnic2

Things to note is the Driver Name/Type, Firmware Version (First Part of it is sufficient), Version (This is the actual driver version on the esxi host).

In the Above screenshot the driver is ‘qedentv’, the firmware version is 8.53.3.0 and the version is 3.11.16.0

Now, we need to find the entries/numbers to search for the exact driver on the VMware compatibility website.

Execute the following command on the ESXI SSH session

vmkchdev -l | grep vmnic2

The highlighted portion is the one we require to search for the driver on VMware Compatibility website

Let us go to the VMware Compatibility website and IO section

We need to fill in the following values —

VID, DID, SVID and Max SSID to get the exact driver for your nic.

Let us fill in the values from our vmkchdev output

VID 1077
DID 8070
SVID 1077
Max SSID 000b

Input the values in VMware IO Compatibility list website

Qlogic Adapter and its versions by vSphere version

Select the vSphere version and click on the version to display the different driver versions we can download

I have selected vSphere version 6.7 U3 in this case and the screenshot is below

The esxi nic driver version and the physical adapter firmware version is different on my Dell server

As you can see, the esxi nic driver version and the physical nic adapter firmware versions are different on this Host. (Typically you should update the esxi nic driver once you upgrade the physical nic firmware as a best practice)

In this case my esxi nic driver version is 3.11.16.0 and the Qlogic NIC Physical firmware version is 8.53.x.x

To download the correct driver, you need to make sure that the esxi nic driver coincides with the Physical nic driver firmware for best compatibility. We will need to download the ‘qedentv’ driver.

We download the driver equal to the physical nic firmware version and the esxi nic driver name which is qedentv in this case

Download the driver.zip file using your my vmware credentials and you can use this zip file in the offline patches in Update Manager to create a baseline for your esxi hosts so this driver can be updated.

NOTE: Put the Host in Maintenance mode before you update the nic driver as this will reboot the esxi host.

VMKPING and its uses in ESXI

May 6, 2021May 6, 2021Pradeep AdapaLeave a comment

I have recently been working with esxi hosts and to decommission them and recommission them into new projects and had to use the command vmkping to test the MTU of certain types of vmkernel ports like VMOTION, VSAN, VTEPs etc.

Here is a refresher for the vmkping commands which are very useful for a day to day Virtual Administrator

Command to check the MTU of 9000 with a certain amount of packets and with a certain interval and using a certain vmkernel port

vmkping -I vmk3 -d -s 8972 -c 1000 -i 0.005

vmkping -d -s 1472 <IP_Address>

In one of the above command vmkernel port is vmk3, for MTU 9000, we will be using 8972 as the packet size , -c is the count of packets and -i is the interval for which the ping will work (In the above example it is 0.005 seconds)

The second command is to test the MTU 1500 and the IP to test. You can also add -I (Interface) and vmkernel port through which you want to ping the IP

Command to check the communication of an IP address through an vmkernel port

vmkping -I vmk# IP address of the host

Command to get all the network adapters and the type of tcp/ip stack assigned to the nics

esxcfg-vmknic -l

Using the above command you can check the netstack which will be used in the below command to ping a vmotion vmkernel port

vmkping -S vmotion -I vmk1 <IP_Address_to_ping>

The -S is for netstack name like vmotion and this is the only command to be used if we use a NetStack

List of arguments:

vmkping [args] [host/IP_Address]

args:

  -4                            use IPv4 (default)

  -6                            use IPv6

  -c <count>            set packet count

  -d                           set DF bit (IPv4) or disable fragmentation (IPv6)

  -D                           vmkernel TCP stack debug mode

  -i <interval>           set interval (secs)

  -I <interface>         outgoing interface – for IPv6 scope or IPv4 bypasses routing lookup

  -N <next_hop>       set IP*_NEXTHOP – bypasses routing lookup

                                  for IPv4, -I option is required

  -s <size>                 set the number of ICMP data bytes to be sent.

                                  The default is 56, which translates to a 64 byte

                                  ICMP frame when added to the 8 byte ICMP header.

                                 (Note: these sizes does not include the IP header).

  -t <ttl>                   set IPv4 Time To Live or IPv6 Hop Limit

  -v                            verbose

  -W <timeout>        set timeout to wait if no responses are received (secs)

  -X                            XML output format for esxcli framework.

  -S                           The network stack instance name. If unspecified the default netstack instance is used.

Install and Configure vRealize Suite Life Cycle Manager 1.2

June 27, 2018August 7, 2018Pradeep Adapa3 Comments

This post details the installation and configuration of the vRealize Suite Life Cycle Manager 1.2 which was recently released by VMware to automatically provision vRA components as part of their Cloud initiative.

First, Download the Life Cycle Manager ova from the vRealize Suite 2017 components and deploy it using the vCenter web client

vRLCM_Installation01

vRLCM_Installation02

vRLCM_Installation03

vRLCM_Installation04

vRLCM_Installation05

vRLCM_Installation06 — Select Enable Content Management option to enable content management.

vRLCM_Installation07

vRLCM_Installation08

vRLCM_Installation09 — Provide the Hostname, default gateway, network IP address, subnet mask, DNS servers and the domain names in this window and click Next to finalize the deployment of the appliance.

vRLCM_Installation10 — Click Finish to finalize the settings and to deploy the Life Cycle Manager Appliance

Once the vm has been deployed and powered ON, you will have to go to a web browser to configure the appliance.

https://IP_Address_of_the_Appliance/vrlcm

vRLCM_Configuration01

use the following credentials to login into the life cycle manager web UI

username: admin@localhost

password: vmware

vRLCM_Configuration02

vRLCM_Configuration03 — The first thing you get after logging into the web UI is to update the root password

vRLCM_Configuration04

Click start to get started with the Life Cycle Manager

vRLCM_Configuration05

vRLCM_Configuration06

vRLCM_Configuration07

vRLCM_Configuration08

vRLCM_Configuration09 — Once you click Next, it will say Done!

Now, we will create a New Environment in the lab

Click on Create Environment option to get started

Once you click on Create Environment option, you will be taken to a tab where it mentions that you will need to take care of a few things before you create the environment.

Let us take care of the Product Binaries first.

Click on Product Binaries option on the tab

vRLCM_Configuration11

vRLCM_Configuration14 — I have used my VMware portal credentials to get the product binaries as I couldn’t get the local and NFS to work to get the product OVA’s.

Once you add the product binaries, let’s go and create a Certificate

vRLCM_Configuration12

vRLCM_Configuration13

Once these two pre-requisites are done, Let us move ahead …

On the main page, click on the Datacenters option on the left-hand side to create a Datacenter before we create the environment

vRLCM_Configuration15

vRLCM_Configuration16 — Click on Add Data Center to provide a name for the Datacenter

vRLCM_Configuration17

Next, we add the vCenter server

vRLCM_Configuration18

vRLCM_Configuration19

vRLCM_Configuration20

Now, Let us go ahead and create an Environment

vRLCM_Configuration21

vRLCM_Configuration22

vRLCM_Configuration23 — Accept the EULA Agreement by scrolling down, once you accept it, the NEXT button will appear

vRLCM_Configuration24

vRLCM_Configuration25

vRLCM_Configuration26

vRLCM_Configuration27

vRLCM_Configuration28 — Provide all the required information. I have provided an existing SQL server and IAAS server, I have used 1 IAAS server for DEM Worker, Orchestrator, Proxy service

vRLCM_Configuration29 — Click on RUN PRE CHECK option to perform the pre-checks before it deploys the environment

vRLCM_Configuration30

In this pre-check, you could get a validation failure which will need to be rectified before you run the pre-check again. its like shown in the picture below

vRLCM_Configuration31

vRLCM_Configuration32

Once you rectify the issue, run the pre-check again

vRLCM_Configuration33

Once the pre-check comes back clean, click on Next to move ahead

vRLCM_Configuration34

Click Submit and the life cycle manager will do the rest.

to check the progress, you can click on Requests icon on the left side of the page and clicking on in progress as the pic below

vRLCM_Configuration35

vRLCM_Configuration36

This process will take a long time … go, get some tea/coffee and it will still be deploying the environment …

vRLCM_Configuration37

vRLCM_Configuration38

This shows how to Install and configure vRealize Life Cycle Management and to create a vRA 7.4 environment.

Install and Configure vSphere Data Protection (VDP) 6.1.8

June 8, 2018Pradeep Adapa1 Comment

In this post, I will be installing and configuring vSphere Data Protection 6.1.8 in vCenter 6.5 Environment

first, login into the vCenter 6.5 web client (flash) and Deploy the ova template and select the VDP.ova file to deploy.

select_vdp_ova_file

ova_template_install_01

ova_template_install_02

ova_template_install_03

ova_template_install_04

ova_template_install_05 — Select the Storage (Datastore) you want to deploy this appliance in and click Next

ova_template_install_06 — Select the Network where you want to Deploy this appliance and click Next

ova_template_install_07 — Provide the Network details such as DNS servers, IP address, Subnet mask, the default gateway of the appliance and click Next

Check all the information entered and click finish for the VM to be deployed in the environment.

Once the appliance is deployed, power in ON

configure_vdp_01

Once the appliance is powered on, go to the web browser to https://Appliance_IP_Address_or_FQDN:8543/vdp-configure to start the configuration of VDP

configure_vdp_02 — In the Web UI, login as root and the default password is *changeme ,* login into the web UI and change the root password

configure_vdp_03

configure_vdp_04 — Check the Network settings as these settings will be automatically populated from the values you set while deploying the appliance

configure_vdp_05

configure_vdp_06 — Provide a new root password for the VDP appliance

vdp_vcenter_registration_01 — Once you enter the details of the External PSC under SSO FQDN you can hit ‘Test Connection’

vdp_vcenter_registration_02

configure_vdp_07 — We now create new storage as this is the first VDP appliance being deployed

configure_vdp_08

configure_vdp_09

configure_vdp_10

configure_vdp_11 — Click Yes to start the configuration

configure_vdp_12 — Once the process completes, the appliance will restart and you can get into the main Web UI

vdp_main_page

vdp_main_UI_vCenter_Home — This is the UI you see when you login into the vCenter Web Client and click on VDP menu item

SSO Server test failed when Configuring VDP 6.1.8 during vCenter Registration

June 7, 2018Pradeep AdapaLeave a comment

Recently, I came across an issue while configuring a new instance of VDP 6.1.8 appliance while performing vCenter Registration to the vCenter appliance 6.5 with an external Platform Services Controller.

below is the error message I have been getting

vdp_error_vcenter_registration

I have provided the administrator account user credentials to the VCSA (vCenter server) with the default ports but still received the error.

Upon some deep troubleshooting, found out that the SSO server is the Platform Services Controller (PSC) since my environment had an external PSC and here is how you resolve this issue:

De-select the checkbox “Use vCenter for SSO authentication”, and add the Platform Services Controller hostname/IP in the new SSO entry line.

vdp_vcenter_registration_01

Now, you can test the connection and it will be a success

vdp_vcenter_registration_02

This is how the issue was resolved. Hope it helps someone out there.

This is with VDP version 6.1.8 connecting to VCSA 6.5 with External PSC

vSphere 6.5 and What’s New?

October 26, 2016Pradeep AdapaLeave a comment

Looks like VMware finally got around to have a public statement on the release of vSphere 6.5 and here are some of its main NEW features

vCenter Server Appliance

vCenter server appliance now has integrated Update Manager
vCenter server appliance now has its native High Availability
vCenter server appliance has better appliance management
vCenter server appliance now has native Backup/Restore

HTML-5 based vSphere web client

Security

VM-level disk encryption capability designed to protect against unauthorized access to data. (This is done using the vSphere storage policy framework)
Encrypted vMotion capability
vSphere 6.5 adds a secure boot to the hypervisor to protect both the hypervisor and guest operating system
Enhanced audit-quality logging capability to provide more information about user actions like who did what, when and where if you need to investigate your environment

Host Resource management

Enhanced Host profiles (updated graphical editor that is part of the vSphere Web Client now has an easy-to-use search function in addition to a new ability to mark individual configuration elements as favorites for quick access, Administrators now have the means to create a hierarchy of host profiles by taking advantage of the new ability to copy settings from one profile to one or many others)
Auto Deploy (Easier to manage in vSphere 6.5 with the introduction of a full-featured graphical interface. Administrators no longer need to use PowerCLI to create and manage deploy rules or custom ESXi images)
Proactive HA ( Proactive HA will detect hardware conditions of a host and allow you to evacuate the VMs before the issue causes an outage. Working in conjunction with participating hardware vendors, vCenter will plug into the hardware monitoring solution to receive the health status of the monitored components such as fans, memory, and power supplies. vSphere can then be configured to respond according to the failure)
vSphere HA Orchestrated Restart (vSphere 6.5 now allows creating dependency chains using VM-to-VM rules. These dependency rules are enforced if when vSphere HA is used to restart VMs from failed hosts. This is great for multi-tier applications that do not recover successfully unless they are restarted in a particular order. A common example to this is a database, app, and web server)
Additional Restart priority levels in HA (vSphere 6.5 adds two additional restart priority levels named Highest and Lowest providing five total. This provides even greater control when planning the recovery of virtual machines managed by vSphere HA)
Simplified vSphere HA Admission Control ( First major change is that the administrator simply needs to define the number of host failures to tolerate (FTT). Once the numbers of hosts are configured, vSphere HA will automatically calculate a percentage of resources to set aside by applying the “Percentage of Cluster Resources” admission control policy. As hosts are added or removed from the cluster, the percentage will be automatically recalculated, Additionally, the vSphere Web Client will issue a warning if vSphere HA detects a host failure would cause a reduction in VM performance based on the actual resource consumption, not only based on the configured reservations)
Fault Tolerance (FT) (vSphere 6.5 FT has more integration with DRS which will help make better placement decisions by ranking the hosts based on the available network bandwidth as well as recommending which datastore to place the secondary vmdk files, FT networks can now be configured to use multiple NICs to increase the overall bandwidth available for FT logging traffic)
DRS Advanced options (VM Distribution, Memory Metric for Load Balancing, CPU over-commitment have their own check boxes in DRS)
Network-Aware DRS (DRS now considers network utilization, DRS observes the Tx and Rx rates of the connected physical uplinks and avoids placing VMs on hosts that are greater than 80% utilized. DRS will not reactively balance the hosts solely based on network utilization, rather, it will use network utilization as an additional check to determine whether the currently selected host is suitable for the VM)