Four Little Diamond ILOs

For a moment there, I had confused ELO and ILO… Things have been pretty busy recently;, so I have not had a chance to publish this post from earlier in the year.

I took delivery of an HP Proliant ML310e in order to consolidate my VMWare VMs into a single place. As part of that I added:

• P410 hardware RAID card as ESXi does not allow the built-in software RAID controller to work
• An HP Advanced ILO4 license. This will allow me to remote onto the console from anywhere in the world, as long as my DD-WRT router is up.
• A Cyber Power CP 1350C UPS which has USB connection and an appliance which one can deploy on ESX to monitor and control the UPS. More info here
• HP’s custom ESX 6.0 Update 02 ISO from here

When I build servers, I always set ILO to use a STATIC IP address, this way, even if the network falls over, as long as you can route traffic the ILO adapter’s NIC, you can fix things. This also holds true if the subnet changs, since you can set a static IP on your laptop or PC and then connect in and fix it.

Step 1: Make sure you install the license.

Without the correct license, ILO will disable after 30 seconds of POST boot time, which stops it being useful for anything beyond fixing BIOS issues.

First off, we boot the ILO instance from cold:

Then boot into HP Smart Update by pressing 4 for Intelligent Provisioning:

A little out-of-place here, but this is the BIOS of the HP for reference:

The server will then run through a number of POST tests and confirm that it is happy with its own hardware:

You will then be taken to a screen advising on boot progress:

Here we can se a screen with the P410 RAID controller setup, in the event you need to configure that.

Eventually, after POST the HP Smart Update OS will boot.

At this point, you can register with HP and/or continue with updates:

I mentioned setting a Static IP on ILO, but I do this after any firmware updates, so let’s proceed on DHCP:

I’ll be provisioning the server with a PXE install of ESXi soon, so for now, select Perform Maintenance:

We’ll do a firmware update first:

The server on this generation will go off to the Internet and check for updates. This is a relatively new feature and I’m glad to say, avoids having to feed CDs into the server to update them. Nice one HP!

My server already had an ILO update waiting, your results will likely vary of course. But let’s install the update:

This is the final point of no return.Going beyond this point upgrades the firmware and CANNOT be undone. I accept this.

The update is applied, and since I’m connect to ILO, I was disconnected while the controller reset. Once that was done, I could reconnect. I simply did nmap -p443 ilo.mydomain.co.uk to check for the reset had completed) and then logged in to see:

A final check, to confirm the ROM version is newer:

Now the firmware is updated, we can proceed with a PXE boot or mount a Virtual CDROM in the server to install ESXi. I will, of course, be using PXE:

I could create a custom PXE menu based on the MAC, and have covered this before, but as I was also testing some new PXE features, I did this manually:

For some reason, VMWare still rely on SYSLINUX 3.86 for their boot loaders, C’Mon Man! So we chainload back to October 2011:

I’ve modified my PXE menu so I know exactly which build and custom image I’m using. More info on that here. So in my case it’s the HP ESXi image:

ESXi PXE boots and as I’m using a scripted install, it proceeds to format the USB key and auto deploy:

After that, it will do the install, keeping caches of the files and saving them to USB for first boot:

The scripted installer also creates a first boot script which sets up NICs, assigns IPs and makes some changes and fixes the ESXi to bring it to the latest build:

After another reboot for the ESXi upgrade, the System boots from USB and boots ESXi:

And now we have an ESXi host provisioned using only ILO. From personal experience, I can confirm that having an ILO license saves a lot of on site time, and lets one make much better use of their resources.

I’ll post again soon with deploying, and then upgrading Acron VMProtect for ESXi.

 

Strawberry Switchblade VMWare Style

I’ve recently updated my Anaconda installer files for ESX. In my preseed file, I now have:


## Create a vswitch named Protected
esxcli network vswitch standard add --ports 128 --vswitch-name Protected
## Now set up management traffic port
esxcli network vswitch standard portgroup add --portgroup-name=ProtectedMgt --vswitch-name=Protected
esxcli network vswitch standard portgroup set --portgroup-name=ProtectedMgt --vlan-id=0
## Set Jumbo frames on the management port
esxcli network ip interface add --interface-name=vmk2 --mtu 9000 --portgroup-name=ProtectedMgt
## Set a static IP so we can get to it if everything is down
esxcli network ip interface ipv4 set --interface-name=vmk2 --ipv4=192.168.100.253 --netmask=255.255.255.0 --type=static
## Add port Group to send traffic around
esxcli network vswitch standard portgroup add --portgroup-name=Protected --vswitch-name=Protected
## Jumbo Frames on the VSwitch
esxcli network vswitch standard set --mtu 9000 --vswitch-name Protected
## Finally, Add a Physical nic (vmnic1) to vSwitch1 (Protected to uplink our VSwitch to vmnic1
esxcli network vswitch standard uplink add --uplink-name=vmnic1 --vswitch-name=Protected


This allows a protected network which is isolated off.

Emergency! Paging Dr. ESX

VMWare have now released Update 02 for ESX 6.0 and HP have also provided their customised image.  I’m now running an ML310e at home, so took the opportunity to update in place using the VSphere offline bundle:

[retrorabble@esx:/vmfs/volumes/raid1] esxcli software vib install -d /vmfs/volumes/raid1/VMware-ESXi-6.0.0-Update2-3620759-HPE-600.U2.9.4.7.13-Mar2016-depot.zip

Installation Result
Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
Reboot Required: true
VIBs Installed: BRCM_bootbank_net-tg3_3.137l.v60.1-1OEM.600.0.0.2494585, EMU_bootbank_elxnet_10.5.121.7-1OEM.600.0.0.2159203, EMU_bootbank_lpfc_10.5.70.0-1OEM.600.0.0.2159203, HPE_bootbank_hpe-build_600.U2.9.4.7.13-2494585, Hewlett-Packard_bootbank_scsi-hpsa_6.0.0.114-1OEM.600.0.0.2494585, Intel_bootbank_net-igb_5.2.10-1OEM.550.0.0.1331820, Intel_bootbank_net-ixgbe_3.21.4.3-1OEM.550.0.0.1331820, LSI_bootbank_scsi-mpt2sas_15.10.06.00.1vmw-1OEM.550.0.0.1198610, MEL_bootbank_nmlx4-core_3.1.0.0-1OEM.600.0.0.2348722, MEL_bootbank_nmlx4-en_3.1.0.0-1OEM.600.0.0.2348722, QLogic_bootbank_misc-cnic-register_1.712.50.v60.1-1OEM.600.0.0.2494585, QLogic_bootbank_net-bnx2_2.2.5j.v60.3-1OEM.600.0.0.2494585, QLogic_bootbank_net-bnx2x_2.712.50.v60.6-1OEM.600.0.0.2494585, QLogic_bootbank_net-cnic_2.712.50.v60.6-1OEM.600.0.0.2494585, QLogic_bootbank_net-nx-nic_6.0.643-1OEM.600.0.0.2494585, QLogic_bootbank_qlnativefc_2.1.27.0-1OEM.600.0.0.2768847, QLogic_bootbank_scsi-bnx2fc_1.712.50.v60.7-1OEM.600.0.0.2494585, QLogic_bootbank_scsi-bnx2i_2.712.50.v60.4-1OEM.600.0.0.2494585, QLogic_bootbank_scsi-qla4xxx_644.6.04.0-1OEM.600.0.0.2159203, VMware_bootbank_ehci-ehci-hcd_1.0-3vmw.600.2.34.3620759, VMware_bootbank_esx-base_6.0.0-2.34.3620759, VMware_bootbank_esx-tboot_6.0.0-2.34.3620759, VMware_bootbank_esx-ui_1.0.0-3617585, VMware_bootbank_misc-drivers_6.0.0-2.34.3620759, VMware_bootbank_net-vmxnet3_1.1.3.0-3vmw.600.2.34.3620759, VMware_bootbank_nvme_1.0e.0.35-1vmw.600.2.34.3620759, VMware_bootbank_sata-ahci_3.0-22vmw.600.2.34.3620759, VMware_bootbank_vsan_6.0.0-2.34.3563498, VMware_bootbank_vsanhealth_6.0.0-3000000.3.0.2.34.3544323, VMware_bootbank_xhci-xhci_1.0-3vmw.600.2.34.3620759, VMware_locker_tools-light_6.0.0-2.34.3620759
VIBs Removed: Hewlett-Packard_bootbank_hp-build_600.9.4.34-2494585, VMware_bootbank_ehci-ehci-hcd_1.0-3vmw.600.1.26.3380124, VMware_bootbank_elxnet_10.2.309.6v-1vmw.600.0.0.2494585, VMware_bootbank_esx-base_6.0.0-1.26.3380124, VMware_bootbank_esx-tboot_6.0.0-0.0.2494585, VMware_bootbank_esx-ui_0.6.0-3623722, VMware_bootbank_lpfc_10.2.309.8-2vmw.600.0.0.2494585, VMware_bootbank_misc-cnic-register_1.78.75.v60.7-1vmw.600.0.0.2494585, VMware_bootbank_misc-drivers_6.0.0-1.26.3380124, VMware_bootbank_net-bnx2_2.2.4f.v60.10-1vmw.600.0.0.2494585, VMware_bootbank_net-bnx2x_1.78.80.v60.12-1vmw.600.0.0.2494585, VMware_bootbank_net-cnic_1.78.76.v60.13-2vmw.600.0.0.2494585, VMware_bootbank_net-igb_5.0.5.1.1-5vmw.600.0.0.2494585, VMware_bootbank_net-ixgbe_3.7.13.7.14iov-20vmw.600.0.0.2494585, VMware_bootbank_net-nx-nic_5.0.621-5vmw.600.0.0.2494585, VMware_bootbank_net-tg3_3.131d.v60.4-2vmw.600.1.26.3380124, VMware_bootbank_net-vmxnet3_1.1.3.0-3vmw.600.0.0.2494585, VMware_bootbank_nmlx4-core_3.0.0.0-1vmw.600.0.0.2494585, VMware_bootbank_nmlx4-en_3.0.0.0-1vmw.600.0.0.2494585, VMware_bootbank_nvme_1.0e.0.35-1vmw.600.1.17.3029758, VMware_bootbank_qlnativefc_2.0.12.0-5vmw.600.0.0.2494585, VMware_bootbank_sata-ahci_3.0-22vmw.600.1.17.3029758, VMware_bootbank_scsi-bnx2fc_1.78.78.v60.8-1vmw.600.0.0.2494585, VMware_bootbank_scsi-bnx2i_2.78.76.v60.8-1vmw.600.0.11.2809209, VMware_bootbank_scsi-hpsa_6.0.0.44-4vmw.600.0.0.2494585, VMware_bootbank_scsi-mpt2sas_19.00.00.00-1vmw.600.0.0.2494585, VMware_bootbank_scsi-qla4xxx_5.01.03.2-7vmw.600.0.0.2494585, VMware_bootbank_vsanhealth_6.0.0-3000000.2.0.1.17.2972216, VMware_bootbank_xhci-xhci_1.0-3vmw.600.1.26.3380124, VMware_locker_tools-light_6.0.0-1.26.3380124
VIBs Skipped: EMU_bootbank_ima-be2iscsi_10.5.101.0-1OEM.600.0.0.2159203, EMU_bootbank_scsi-be2iscsi_10.5.101.0-1OEM.600.0.0.2159203, Hewlett-Packard_bootbank_char-hpcru_6.0.6.14-1OEM.600.0.0.2159203, Hewlett-Packard_bootbank_char-hpilo_600.9.0.2.8-1OEM.600.0.0.2159203, Hewlett-Packard_bootbank_hp-ams_600.10.3.0-15.2494585, Hewlett-Packard_bootbank_hp-conrep_6.0.0.1-0.0.13.2159203, Hewlett-Packard_bootbank_hp-esxi-fc-enablement_600.2.4.6-2494585, Hewlett-Packard_bootbank_hp-smx-provider_600.03.09.00.15-2768847, Hewlett-Packard_bootbank_hpbootcfg_6.0.0.02-01.00.11.2159203, Hewlett-Packard_bootbank_hpnmi_600.2.3.14-2159203, Hewlett-Packard_bootbank_hponcfg_6.0.0.04-00.13.17.2159203, Hewlett-Packard_bootbank_hpssacli_2.30.6.0-6.0.0.2159203, Hewlett-Packard_bootbank_hptestevent_6.0.0.01-00.00.8.2159203, Hewlett-Packard_bootbank_scsi-hpdsa_5.5.0.46-1OEM.550.0.0.1331820, Hewlett-Packard_bootbank_scsi-hpvsa_5.5.0.100-1OEM.550.0.0.1331820, Intel_bootbank_intelcim-provider_0.5-1.4, Intel_bootbank_net-i40e_1.2.48-1OEM.550.0.0.1331820, MEL_bootbank_nmst_4.0.0.20-1OEM.600.0.0.2295424, QLogic_bootbank_net-qlcnic_6.1.191-1OEM.600.0.0.2494585, QLogic_bootbank_scsi-bfa_3.2.5.0-1OEM.550.0.0.1331820, VMWARE_bootbank_mtip32xx-native_3.8.5-1vmw.600.0.0.2494585, VMware_bootbank_ata-pata-amd_0.3.10-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-atiixp_0.4.6-4vmw.600.0.0.2494585, VMware_bootbank_ata-pata-cmd64x_0.2.5-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-hpt3x2n_0.3.4-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-pdc2027x_1.0-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-serverworks_0.4.3-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-sil680_0.4.8-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-via_0.3.3-2vmw.600.0.0.2494585, VMware_bootbank_block-cciss_3.6.14-10vmw.600.0.0.2494585, VMware_bootbank_cpu-microcode_6.0.0-0.0.2494585, VMware_bootbank_emulex-esx-elxnetcli_10.2.309.6v-0.0.2494585, VMware_bootbank_esx-dvfilter-generic-fastpath_6.0.0-0.0.2494585, VMware_bootbank_esx-xserver_6.0.0-0.0.2494585, VMware_bootbank_ima-qla4xxx_2.02.18-1vmw.600.0.0.2494585, VMware_bootbank_ipmi-ipmi-devintf_39.1-4vmw.600.0.0.2494585, VMware_bootbank_ipmi-ipmi-msghandler_39.1-4vmw.600.0.0.2494585, VMware_bootbank_ipmi-ipmi-si-drv_39.1-4vmw.600.0.0.2494585, VMware_bootbank_lsi-mr3_6.605.08.00-7vmw.600.1.17.3029758, VMware_bootbank_lsi-msgpt3_06.255.12.00-8vmw.600.1.17.3029758, VMware_bootbank_lsu-hp-hpsa-plugin_1.0.0-1vmw.600.0.0.2494585, VMware_bootbank_lsu-lsi-lsi-mr3-plugin_1.0.0-2vmw.600.0.11.2809209, VMware_bootbank_lsu-lsi-lsi-msgpt3-plugin_1.0.0-1vmw.600.0.0.2494585, VMware_bootbank_lsu-lsi-megaraid-sas-plugin_1.0.0-2vmw.600.0.11.2809209, VMware_bootbank_lsu-lsi-mpt2sas-plugin_1.0.0-4vmw.600.1.17.3029758, VMware_bootbank_lsu-lsi-mptsas-plugin_1.0.0-1vmw.600.0.0.2494585, VMware_bootbank_net-e1000_8.0.3.1-5vmw.600.0.0.2494585, VMware_bootbank_net-e1000e_3.2.2.1-1vmw.600.1.26.3380124, VMware_bootbank_net-enic_2.1.2.38-2vmw.600.0.0.2494585, VMware_bootbank_net-forcedeth_0.61-2vmw.600.0.0.2494585, VMware_bootbank_net-mlx4-core_1.9.7.0-1vmw.600.0.0.2494585, VMware_bootbank_net-mlx4-en_1.9.7.0-1vmw.600.0.0.2494585, VMware_bootbank_nmlx4-rdma_3.0.0.0-1vmw.600.0.0.2494585, VMware_bootbank_ohci-usb-ohci_1.0-3vmw.600.0.0.2494585, VMware_bootbank_rste_2.0.2.0088-4vmw.600.0.0.2494585, VMware_bootbank_sata-ata-piix_2.12-10vmw.600.0.0.2494585, VMware_bootbank_sata-sata-nv_3.5-4vmw.600.0.0.2494585, VMware_bootbank_sata-sata-promise_2.12-3vmw.600.0.0.2494585, VMware_bootbank_sata-sata-sil24_1.1-1vmw.600.0.0.2494585, VMware_bootbank_sata-sata-sil_2.3-4vmw.600.0.0.2494585, VMware_bootbank_sata-sata-svw_2.3-3vmw.600.0.0.2494585, VMware_bootbank_scsi-aacraid_1.1.5.1-9vmw.600.0.0.2494585, VMware_bootbank_scsi-adp94xx_1.0.8.12-6vmw.600.0.0.2494585, VMware_bootbank_scsi-aic79xx_3.1-5vmw.600.0.0.2494585, VMware_bootbank_scsi-fnic_1.5.0.45-3vmw.600.0.0.2494585, VMware_bootbank_scsi-ips_7.12.05-4vmw.600.0.0.2494585, VMware_bootbank_scsi-megaraid-mbox_2.20.5.1-6vmw.600.0.0.2494585, VMware_bootbank_scsi-megaraid-sas_6.603.55.00-2vmw.600.0.0.2494585, VMware_bootbank_scsi-megaraid2_2.00.4-9vmw.600.0.0.2494585, VMware_bootbank_scsi-mptsas_4.23.01.00-9vmw.600.0.0.2494585, VMware_bootbank_scsi-mptspi_4.23.01.00-9vmw.600.0.0.2494585, VMware_bootbank_uhci-usb-uhci_1.0-3vmw.600.0.0.2494585


Now we have upgraded, we’ll need to reboot. This varies depending on your estate, but with an ILO license, I culd sit and watch ESX progress back up again.

WebUI

One of the more interesting features of Update 02 is that it now includes the VSphere WebUI client by default. I have been using this for some time, and have been generally impressed by the bug fixes and updates it receives. It can be accessed at https://ip_of_your_server/ui so do be aware of this in a security context, and limit down the firewalls if needed.

Upon first log in, you are greeted with a message on the support level of this ESX host. In my case:

You are running HPE Customized Image ESXi 6.0.0 Update 2 version 600.U2.9.4.7.13 released on March 2016 and based on ESXi 6.0.0 Update 2 Vmkernel Release Build 3620759.

When I make changes, and on a regular basis, I like to make backups of the ESX config. This means we can restore fairly quickly. To do this, one can:

retrorabble@puppet:[~]$ ssh root@esx

The time and date of this login have been sent to the system logs.

VMware offers supported, powerful system administration tools.  Please
see www.vmware.com/go/sysadmintools for details.

The ESXi Shell can be disabled by an administrative user. See the
vSphere Security documentation for more information.
[root@esx:~] vim-cmd hostsvc/firmware/sync_config ; vim-cmd hostsvc/firmware/backup_config
Bundle can be downloaded at : http://*/downloads/520f9ab6-eb37-6eb0-4a3f-783cadf9f0a8/configBundle-esx.tgz
[root@esx:~] Connection to esx closed.

Now we can take a backup from our machine:

retrorabble@puppet:[~]$ wget --no-check-certificate http://esx.yourdomain.tld/downloads/520f9ab6-eb37-6eb0-4a3f-783cadf9f0a8/configBundle-esx.tgz -O esx.yourdomain.tld_configBundle-esx_backup2016-03-29.tgz
--2016-03-29 19:07:13--  http://esx.yourdomain.tld/downloads/520f9ab6-eb37-6eb0-4a3f-783cadf9f0a8/configBundle-esx.tgz
Resolving esx.yourdomain.tld (esx.yourdomain.tld)... 192.168.139.3
Connecting to esx.yourdomain.tld (esx.yourdomain.tld)|192.168.139.3|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://esx.yourdomain.tld/downloads/520f9ab6-eb37-6eb0-4a3f-783cadf9f0a8/configBundle-esx.tgz [following]
--2016-03-29 19:07:13--  https://esx.yourdomain.tld/downloads/520f9ab6-eb37-6eb0-4a3f-783cadf9f0a8/configBundle-esx.tgz
Connecting to esx.yourdomain.tld (esx.yourdomain.tld)|192.168.139.3|:443... connected.
WARNING: cannot verify esx.yourdomain.tld's certificate, issued by '/O=VMware Installer':
Unable to locally verify the issuer's authority.
WARNING: no certificate subject alternative name matches
requested host name 'esx.yourdomain.tld'.
HTTP request sent, awaiting response... 200 OK
Length: 25019 (24K) [application/x-tar]
Saving to: 'esx.yourdomain.tld_configBundle-esx_backup2016-03-29.tgz'

100%[========================================================================>] 25,019      –.-K/s   in 0s

2016-03-29 19:07:13 (338 MB/s) - 'esx.yourdomain.tld_configBundle-esx_backup2016-03-29.tgz' saved [25019/25019]

Which allows the file to be copied over to some backup space easily. However, one does need to ensure SSH keys are up-to-date for scripting this.

New Kids On The (Reserved) Block

Yeah, off the back off that VMWare CBT issue, I’m making incredibly lame puns, but tough!

Ubuntu preseed files are something I’ve blogged about before, and recently, I finally got some time to work on the issue where Linux will use by default, 5% of a disk for reserved blocks.

I think this is a little excessive, and like to keep 1% for those total emergencies when there no LVM space or downtime is not possible. Think still gives plenty of slack and with Zabbix keeping an eye on disk space in general, it’s really just to satisfy my OCD…

In order to avoid having to do manual work on server provisioning, when my time could be better spent listening to Then Jericho and reading Private Eye, I added the following stanza in our preseed’s partition recipie:

# Updated: 07/01/2016 v2.1 retrorabble      Reduce reserved blocks from 5% to 1%   #
reserved_for_root{ 1 }           \
1024 40 1024 ext4                \
$defaultignore{ }                \
$lvmok{ }                        \
lv_name{ home }                  \
method{ format }                 \
format{ }                        \
reserved_for_root{ 1 }           \
use_filesystem{ }                \
filesystem{ ext4 }               \
mountpoint{ /home }              \
.           .                    \


This will need added in each partition/LV you plan on creating. I’ve highlighed the line too, since positioning is important in the preseed.

On and on and on and VMWare CBT

So VMWare have released yet another fix to Update 3b. This is (yet another) attempt to get CBT bugs ironed out.

I really do hope the QA department of VMWare made a New Year resolution to stop fundamentally breaking simple functions in the software!

More upgrades of the estate are required, and once again, it requires a reboot, and even more downtime. Luckily, patch Tuesday is next week, so you might be able to schedule it around that. Here’s how to patch:

$ ssh root@dhcp03
VMware offers supported, powerful system administration tools.  Please see www.vmware.com/go/sysadmintools for details.

The ESXi Shell can be disabled by an administrative user. See the vSphere Security documentation for more information.

[root@dhcp03:~] vmware -v
VMware ESXi 6.0.0 build-3073146

[root@dhcp03:~] esxcli software acceptance set --level=PartnerSupported
[root@dhcp03:~] esxcli network firewall ruleset set -e true -r httpClient
[root@dhcp03:~] ## esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-6.0.0-20160104001-standard
Update Result
Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
Reboot Required: true
VIBs Installed: VMware_bootbank_ehci-ehci-hcd_1.0-3vmw.600.1.26.3380124, VMware_bootbank_esx-base_6.0.0-1.26.3380124, VMware_bootbank_misc-drivers_6.0.0-1.26.3380124, VMware_bootbank_net-e1000e_3.2.2.1-1vmw.600.1.26.3380124, VMware_bootbank_net-tg3_3.131d.v60.4-2vmw.600.1.26.3380124, VMware_bootbank_xhci-xhci_1.0-3vmw.600.1.26.3380124, VMware_locker_tools-light_6.0.0-1.26.3380124
VIBs Removed: VMware_bootbank_ehci-ehci-hcd_1.0-3vmw.600.0.0.2494585, VMware_bootbank_esx-base_6.0.0-1.20.3073146, VMware_bootbank_misc-drivers_6.0.0-1.17.3029758, VMware_bootbank_net-e1000e_2.5.4-6vmw.600.0.0.2494585, VMware_bootbank_net-tg3_3.131d.v60.4-1vmw.600.0.0.2494585, VMware_bootbank_xhci-xhci_1.0-2vmw.600.1.17.3029758, VMware_locker_tools-light_6.0.0-1.17.3029758
VIBs Skipped: VMWARE_bootbank_mtip32xx-native_3.8.5-1vmw.600.0.0.2494585, VMware_bootbank_ata-pata-amd_0.3.10-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-atiixp_0.4.6-4vmw.600.0.0.2494585, VMware_bootbank_ata-pata-cmd64x_0.2.5-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-hpt3x2n_0.3.4-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-pdc2027x_1.0-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-serverworks_0.4.3-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-sil680_0.4.8-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-via_0.3.3-2vmw.600.0.0.2494585, VMware_bootbank_block-cciss_3.6.14-10vmw.600.0.0.2494585, VMware_bootbank_cpu-microcode_6.0.0-0.0.2494585, VMware_bootbank_elxnet_10.2.309.6v-1vmw.600.0.0.2494585, VMware_bootbank_emulex-esx-elxnetcli_10.2.309.6v-0.0.2494585, VMware_bootbank_esx-dvfilter-generic-fastpath_6.0.0-0.0.2494585, VMware_bootbank_esx-tboot_6.0.0-0.0.2494585, VMware_bootbank_esx-xserver_6.0.0-0.0.2494585, VMware_bootbank_ima-qla4xxx_2.02.18-1vmw.600.0.0.2494585, VMware_bootbank_ipmi-ipmi-devintf_39.1-4vmw.600.0.0.2494585, VMware_bootbank_ipmi-ipmi-msghandler_39.1-4vmw.600.0.0.2494585, VMware_bootbank_ipmi-ipmi-si-drv_39.1-4vmw.600.0.0.2494585, VMware_bootbank_lpfc_10.2.309.8-2vmw.600.0.0.2494585, VMware_bootbank_lsi-mr3_6.605.08.00-7vmw.600.1.17.3029758, VMware_bootbank_lsi-msgpt3_06.255.12.00-8vmw.600.1.17.3029758, VMware_bootbank_lsu-hp-hpsa-plugin_1.0.0-1vmw.600.0.0.2494585, VMware_bootbank_lsu-lsi-lsi-mr3-plugin_1.0.0-2vmw.600.0.11.2809209, VMware_bootbank_lsu-lsi-lsi-msgpt3-plugin_1.0.0-1vmw.600.0.0.2494585, VMware_bootbank_lsu-lsi-megaraid-sas-plugin_1.0.0-2vmw.600.0.11.2809209, VMware_bootbank_lsu-lsi-mpt2sas-plugin_1.0.0-4vmw.600.1.17.3029758, VMware_bootbank_lsu-lsi-mptsas-plugin_1.0.0-1vmw.600.0.0.2494585, VMware_bootbank_misc-cnic-register_1.78.75.v60.7-1vmw.600.0.0.2494585, VMware_bootbank_net-bnx2_2.2.4f.v60.10-1vmw.600.0.0.2494585, VMware_bootbank_net-bnx2x_1.78.80.v60.12-1vmw.600.0.0.2494585, VMware_bootbank_net-cnic_1.78.76.v60.13-2vmw.600.0.0.2494585, VMware_bootbank_net-e1000_8.0.3.1-5vmw.600.0.0.2494585, VMware_bootbank_net-enic_2.1.2.38-2vmw.600.0.0.2494585, VMware_bootbank_net-forcedeth_0.61-2vmw.600.0.0.2494585, VMware_bootbank_net-igb_5.0.5.1.1-5vmw.600.0.0.2494585, VMware_bootbank_net-ixgbe_3.7.13.7.14iov-20vmw.600.0.0.2494585, VMware_bootbank_net-mlx4-core_1.9.7.0-1vmw.600.0.0.2494585, VMware_bootbank_net-mlx4-en_1.9.7.0-1vmw.600.0.0.2494585, VMware_bootbank_net-nx-nic_5.0.621-5vmw.600.0.0.2494585, VMware_bootbank_net-vmxnet3_1.1.3.0-3vmw.600.0.0.2494585, VMware_bootbank_nmlx4-core_3.0.0.0-1vmw.600.0.0.2494585, VMware_bootbank_nmlx4-en_3.0.0.0-1vmw.600.0.0.2494585, VMware_bootbank_nmlx4-rdma_3.0.0.0-1vmw.600.0.0.2494585, VMware_bootbank_nvme_1.0e.0.35-1vmw.600.1.17.3029758, VMware_bootbank_ohci-usb-ohci_1.0-3vmw.600.0.0.2494585, VMware_bootbank_qlnativefc_2.0.12.0-5vmw.600.0.0.2494585, VMware_bootbank_rste_2.0.2.0088-4vmw.600.0.0.2494585, VMware_bootbank_sata-ahci_3.0-22vmw.600.1.17.3029758, VMware_bootbank_sata-ata-piix_2.12-10vmw.600.0.0.2494585, VMware_bootbank_sata-sata-nv_3.5-4vmw.600.0.0.2494585, VMware_bootbank_sata-sata-promise_2.12-3vmw.600.0.0.2494585, VMware_bootbank_sata-sata-sil24_1.1-1vmw.600.0.0.2494585, VMware_bootbank_sata-sata-sil_2.3-4vmw.600.0.0.2494585, VMware_bootbank_sata-sata-svw_2.3-3vmw.600.0.0.2494585, VMware_bootbank_scsi-aacraid_1.1.5.1-9vmw.600.0.0.2494585, VMware_bootbank_scsi-adp94xx_1.0.8.12-6vmw.600.0.0.2494585, VMware_bootbank_scsi-aic79xx_3.1-5vmw.600.0.0.2494585, VMware_bootbank_scsi-bnx2fc_1.78.78.v60.8-1vmw.600.0.0.2494585, VMware_bootbank_scsi-bnx2i_2.78.76.v60.8-1vmw.600.0.11.2809209, VMware_bootbank_scsi-fnic_1.5.0.45-3vmw.600.0.0.2494585, VMware_bootbank_scsi-hpsa_6.0.0.44-4vmw.600.0.0.2494585, VMware_bootbank_scsi-ips_7.12.05-4vmw.600.0.0.2494585, VMware_bootbank_scsi-megaraid-mbox_2.20.5.1-6vmw.600.0.0.2494585, VMware_bootbank_scsi-megaraid-sas_6.603.55.00-2vmw.600.0.0.2494585, VMware_bootbank_scsi-megaraid2_2.00.4-9vmw.600.0.0.2494585, VMware_bootbank_scsi-mpt2sas_19.00.00.00-1vmw.600.0.0.2494585, VMware_bootbank_scsi-mptsas_4.23.01.00-9vmw.600.0.0.2494585, VMware_bootbank_scsi-mptspi_4.23.01.00-9vmw.600.0.0.2494585, VMware_bootbank_scsi-qla4xxx_5.01.03.2-7vmw.600.0.0.2494585, VMware_bootbank_uhci-usb-uhci_1.0-3vmw.600.0.0.2494585, VMware_bootbank_vsanhealth_6.0.0-3000000.2.0.1.17.2972216

[root@dhcp03:~] vmware -v
VMware ESXi 6.0.0 build-3380124

Things are getting to be so bad with VMWare, we now turn off CBT in Acronis Backup and live with slightly longer backup times because I feel we can no longer trust VMWare not to mess up something again. We rolled this out after my last port, and I’ve now been vindicated.

Unforutnately, now I cannot now get this out of my head…

Rip it up and start VMWare again

So VMWare have released a fix to Update 03 since my last post. Once again the QA department of a company are away with the fairies…

So to upgrade your estate, run the following:

root@esx55test[~]# esxcli network firewall ruleset set -e true -r httpClient
root@esx55test[~]# esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-5.5.0-20151004001-standard
Update Result
Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
Reboot Required: true
VIBs Installed: VMware_bootbank_esx-base_5.5.0-3.71.3116895
VIBs Removed: VMware_bootbank_esx-base_5.5.0-3.68.3029944
VIBs Skipped: VMware_bootbank_ata-pata-amd_0.3.10-3vmw.550.0.0.1331820, VMware_bootbank_ata-pata-atiixp_0.4.6-4vmw.550.0.0.1331820, VMware_bootbank_ata-pata-cmd64x_0.2.5-3vmw.550.0.0.1331820, VMware_bootbank_ata-pata-hpt3x2n_0.3.4-3vmw.550.0.0.1331820, VMware_bootbank_ata-pata-pdc2027x_1.0-3vmw.550.0.0.1331820, VMware_bootbank_ata-pata-serverworks_0.4.3-3vmw.550.0.0.1331820, VMware_bootbank_ata-pata-sil680_0.4.8-3vmw.550.0.0.1331820, VMware_bootbank_ata-pata-via_0.3.3-2vmw.550.0.0.1331820, VMware_bootbank_block-cciss_3.6.14-10vmw.550.0.0.1331820, VMware_bootbank_ehci-ehci-hcd_1.0-3vmw.550.0.0.1331820, VMware_bootbank_elxnet_10.2.309.6v-1vmw.550.3.68.3029944, VMware_bootbank_esx-dvfilter-generic-fastpath_5.5.0-0.0.1331820, VMware_bootbank_esx-tboot_5.5.0-2.33.2068190, VMware_bootbank_esx-xlibs_5.5.0-0.0.1331820, VMware_bootbank_esx-xserver_5.5.0-0.0.1331820, VMware_bootbank_ima-qla4xxx_2.01.31-1vmw.550.0.0.1331820, VMware_bootbank_ipmi-ipmi-devintf_39.1-4vmw.550.0.0.1331820, VMware_bootbank_ipmi-ipmi-msghandler_39.1-4vmw.550.0.0.1331820, VMware_bootbank_ipmi-ipmi-si-drv_39.1-4vmw.550.0.0.1331820, VMware_bootbank_lpfc_10.0.100.1-1vmw.550.0.0.1331820, VMware_bootbank_lsi-mr3_0.255.03.01-2vmw.550.3.68.3029944, VMware_bootbank_lsi-msgpt3_00.255.03.03-1vmw.550.1.15.1623387, VMware_bootbank_misc-cnic-register_1.72.1.v50.1i-1vmw.550.0.0.1331820, VMware_bootbank_misc-drivers_5.5.0-3.68.3029944, VMware_bootbank_mtip32xx-native_3.3.4-1vmw.550.1.15.1623387, VMware_bootbank_net-be2net_4.6.100.0v-1vmw.550.0.0.1331820, VMware_bootbank_net-bnx2_2.2.3d.v55.2-1vmw.550.0.0.1331820, VMware_bootbank_net-bnx2x_1.72.56.v55.2-1vmw.550.0.0.1331820, VMware_bootbank_net-cnic_1.72.52.v55.1-1vmw.550.0.0.1331820, VMware_bootbank_net-e1000_8.0.3.1-3vmw.550.0.0.1331820, VMware_bootbank_net-e1000e_3.2.2.1-2vmw.550.3.68.3029944, VMware_bootbank_net-enic_1.4.2.15a-1vmw.550.0.0.1331820, VMware_bootbank_net-forcedeth_0.61-2vmw.550.0.0.1331820, VMware_bootbank_net-igb_5.0.5.1.1-1vmw.550.2.54.2403361, VMware_bootbank_net-ixgbe_3.7.13.7.14iov-12vmw.550.2.62.2718055, VMware_bootbank_net-mlx4-core_1.9.7.0-1vmw.550.0.0.1331820, VMware_bootbank_net-mlx4-en_1.9.7.0-1vmw.550.0.0.1331820, VMware_bootbank_net-nx-nic_5.0.621-1vmw.550.0.0.1331820, VMware_bootbank_net-tg3_3.123c.v55.5-1vmw.550.2.33.2068190, VMware_bootbank_net-vmxnet3_1.1.3.0-3vmw.550.2.39.2143827, VMware_bootbank_ohci-usb-ohci_1.0-3vmw.550.0.0.1331820, VMware_bootbank_qlnativefc_1.0.12.0-1vmw.550.0.0.1331820, VMware_bootbank_rste_2.0.2.0088-4vmw.550.1.15.1623387, VMware_bootbank_sata-ahci_3.0-22vmw.550.3.68.3029944, VMware_bootbank_sata-ata-piix_2.12-10vmw.550.2.33.2068190, VMware_bootbank_sata-sata-nv_3.5-4vmw.550.0.0.1331820, VMware_bootbank_sata-sata-promise_2.12-3vmw.550.0.0.1331820, VMware_bootbank_sata-sata-sil24_1.1-1vmw.550.0.0.1331820, VMware_bootbank_sata-sata-sil_2.3-4vmw.550.0.0.1331820, VMware_bootbank_sata-sata-svw_2.3-3vmw.550.0.0.1331820, VMware_bootbank_scsi-aacraid_1.1.5.1-9vmw.550.0.0.1331820, VMware_bootbank_scsi-adp94xx_1.0.8.12-6vmw.550.0.0.1331820, VMware_bootbank_scsi-aic79xx_3.1-5vmw.550.0.0.1331820, VMware_bootbank_scsi-bnx2fc_1.72.53.v55.1-1vmw.550.0.0.1331820, VMware_bootbank_scsi-bnx2i_2.72.11.v55.4-1vmw.550.0.0.1331820, VMware_bootbank_scsi-fnic_1.5.0.4-1vmw.550.0.0.1331820, VMware_bootbank_scsi-hpsa_5.5.0-44vmw.550.0.0.1331820, VMware_bootbank_scsi-ips_7.12.05-4vmw.550.0.0.1331820, VMware_bootbank_scsi-lpfc820_8.2.3.1-129vmw.550.0.0.1331820, VMware_bootbank_scsi-megaraid-mbox_2.20.5.1-6vmw.550.0.0.1331820, VMware_bootbank_scsi-megaraid-sas_5.34-9vmw.550.3.68.3029944, VMware_bootbank_scsi-megaraid2_2.00.4-9vmw.550.0.0.1331820, VMware_bootbank_scsi-mpt2sas_14.00.00.00-3vmw.550.3.68.3029944, VMware_bootbank_scsi-mptsas_4.23.01.00-9vmw.550.3.68.3029944, VMware_bootbank_scsi-mptspi_4.23.01.00-9vmw.550.3.68.3029944, VMware_bootbank_scsi-qla2xxx_902.k1.1-12vmw.550.3.68.3029944, VMware_bootbank_scsi-qla4xxx_5.01.03.2-6vmw.550.0.0.1331820, VMware_bootbank_uhci-usb-uhci_1.0-3vmw.550.0.0.1331820, VMware_bootbank_xhci-xhci_1.0-2vmw.550.3.68.3029944, VMware_locker_tools-light_5.5.0-3.68.3029944
root@esx55test[~]# vmware -v
VMware ESXi 5.5.0 build-3029944
root@esx55test[~]#

After reboot:

root@esx55test[~]# vmware -v
VMware ESXi 5.5.0 build-3116895

If you are using Zabbix, create an Trigger with:

Name: VMWare ESXi is out of date on {HOST.NAME} (build is {ITEM.LASTVALUE1}) (SNAPSHOT)
Expression: {Template SNMP OS ESXi:vmwProdBuild.0.last()}<3116895
Description: Checks that the version of VMWare 5.5 is not the version which bombs out on Error 11 with snapshot consolidation. c.f: http://kb.vmware.com/kb/2133825
Severity: HIGH
Enabled: [X]

I set this to HIGH since it can cause problems making backups and also keeps VMs running on Snapshots, which is a bad thing…

Let’s hope someone in the QA department was given a slap for that one!

Everybody Have PXE Tonight

OK, so that’s a little bit of artistic license with the Wang Chung reference, but hey ho.

Recenrly, I had to upgrade our base ESX images on the PXE server because VMWare helpfully sneaked in a change to TPS with a security fix (gee thanks guys! After all, adding major changes in policy to point releases made sense to some committee of managers, right…?)

So we simply added:

/bin/esxcfg-advcfg --set 0 /Mem/ShareForceSalting

To the %firstboot section of our preseed file. However, the build of 5.5 needs to be on build 2718055 or newer, and 6.0 needs to be on 2809111 or newer. This meant that in order to avoid having to do two reboots, I though it was time to upgrade our base install media on PXE to be 2718055 (we could make it a newer build, but we do an in-place upgrade to using the %firstboot section) which allows the function to work.

I’ll presume you are using puppet, and I’m going to show you how to add the HP custom ESX 6.0 image to a PXE server here.

Step 1: We get all the needed files in one place, including the now outdated SYSLINUX version VMWare insist on using:

gsuser@tftp:[~]$ cd /tmp/
gsuser@tftp:[/tmp]$ ls
syslinux-3.86.tar.gz VMware-ESXi-6.0.0-2494585-HP-600.9.3.30.2-Jun2015.iso
gsuser@tftp:[/tmp]$ sudo mount -o loop,ro VMware-ESXi-6.0.0-2494585-HP-600.9.3.30.2-Jun2015.iso /mnt/
gsuser@tftp:[/tmp]$ tar zxf syslinux-3.86.tar.gz
gsuser@tftp:[/tmp]$ mkdir -p /tftpboot/esx60/esxi-6.0.0-2494585-standard_hp

Step 2: we copy our files to the right place, and confirm it looks right:
gsuser@tftp:[/tmp]$ cp syslinux-3.86/core/pxelinux.0 /tftpboot/esx60
gsuser@tftp:[/tmp]$ cp syslinux-3.86/com32/menu/menu.c32 /tftpboot/esx60
gsuser@tftp:[/tmp]$ cp syslinux-3.86/com32/modules/chain.c32 /tftpboot/esx60
gsuser@tftp:[/tmp]$ cd /tftpboot/esxi60
gsuser@tftp:[/tftpboot/esxi60]$ rsync -a /mnt/ ./esxi-6.0.0-2494585-standard_hp/
gsuser@tftp:[/tftpboot/esxi60]$ cd esxi-6.0.0-2494585-standard_hp/
gsuser@tftp:[/tftpboot/esxi60/esxi-6.0.0-2494585-standard_hp]$ ls
a.b00 efi ima_be2i.v00 lsu_lsi_.v01 net_i40e.v00 safeboot.c32 scsi_fni.v00 uc_intel.b00
ata_pata.v00 efiboot.img ima_qla4.v00 lsu_lsi_.v02 net_igb.v00 sata_ahc.v00 scsi_hpd.v00 uhci_usb.v00
ata_pata.v01 ehci_ehc.v00 imgdb.tgz lsu_lsi_.v03 net_ixgb.v00 sata_ata.v00 scsi_hps.v00 upgrade
ata_pata.v02 elxnet.v00 imgpayld.tgz lsu_lsi_.v04 net_mlx4.v00 sata_sat.v00 scsi_hpv.v00 user.b00
ata_pata.v03 emulex_e.v00 ipmi_ipm.v00 mboot.c32 net_mlx4.v01 sata_sat.v01 scsi_ips.v00 useropts.gz
ata_pata.v04 esx_dvfi.v00 ipmi_ipm.v01 menu.c32 net_nx_n.v00 sata_sat.v02 scsi_meg.v00 vmware-esx-base-osl.txt
ata_pata.v05 hp_ams.v00 ipmi_ipm.v02 misc_cni.v00 net_qlcn.v00 sata_sat.v03 scsi_meg.v01 vmware-esx-base-readme
ata_pata.v06 hpbootcf.v00 isolinux.bin misc_dri.v00 net_tg3.v00 sata_sat.v04 scsi_meg.v02 weaselin.t00
ata_pata.v07 hp_build.v00 isolinux.cfg mtip32xx.v00 net_vmxn.v00 sb.v00 scsi_mpt.v00 xhci_xhc.v00
b.b00 hp_conre.v00 jumpstrt.gz net_bnx2.v00 nmlx4_co.v00 scsi_aac.v00 scsi_mpt.v01 xorg.v00
block_cc.v00 hp_esxi_.v00 k.b00 net_bnx2.v01 nmlx4_en.v00 scsi_adp.v00 scsi_mpt.v02
boot.cat hpnmi.v00 lpfc.v00 net_cnic.v00 nmlx4_rd.v00 scsi_aic.v00 scsi_qla.v00
boot.cfg hponcfg.v00 lsi_mr3.v00 net_e100.v00 nvme.v00 scsi_be2.v00 s.v00
chardevs.b00 hp_smx_p.v00 lsi_msgp.v00 net_e100.v01 ohci_usb.v00 scsi_bfa.v00 tboot.b00
char_hpc.v00 hpssacli.v00 lsu_hp_h.v00 net_enic.v00 qlnative.v00 scsi_bnx.v00 tools.t00
char_hpi.v00 hptestev.v00 lsu_lsi_.v00 net_forc.v00 rste.v00 scsi_bnx.v01 uc_amd.b00

Step 3: So we now need to update the boot.cfg file, adding in a prefix. This reduces the typing and makes it possible to have a nice clean TFTP root:
gsuser@tftp:[/tftpboot/esxi60]$ cat esxi-6.0.0-2494585-standard_hp/boot.cfg
bootstate=0
title=Loading ESXi installer (HP Image)
timeout=5
prefix=esxi-6.0.0-2494585-standard_hp
kernel=tboot.b00
kernelopt=runweasel
modules=b.b00 --- jumpstrt.gz --- useropts.gz --- k.b00 --- chardevs.b00 --- a.b00 --- user.b00 --- uc_intel.b00 --- uc_amd.b00 --- sb.v00 --- s.v00 --- misc_cni.v00 --- net_bnx2.v00 --- net_bnx2.v01 --- net_cnic.v00 --- net_tg3.v00 --- scsi_bnx.v00 --- scsi_bnx.v01 --- lpfc.v00 --- elxnet.v00 --- ima_be2i.v00 --- scsi_be2.v00 --- char_hpc.v00 --- char_hpi.v00 --- hp_ams.v00 --- hp_build.v00 --- hp_conre.v00 --- hp_esxi_.v00 --- hp_smx_p.v00 --- hpbootcf.v00 --- hpnmi.v00 --- hponcfg.v00 --- hpssacli.v00 --- hptestev.v00 --- scsi_hpd.v00 --- scsi_hps.v00 --- scsi_hpv.v00 --- net_i40e.v00 --- net_igb.v00 --- net_ixgb.v00 --- scsi_mpt.v00 --- ima_qla4.v00 --- net_nx_n.v00 --- net_qlcn.v00 --- qlnative.v00 --- scsi_bfa.v00 --- scsi_qla.v00 --- mtip32xx.v00 --- ata_pata.v00 --- ata_pata.v01 --- ata_pata.v02 --- ata_pata.v03 --- ata_pata.v04 --- ata_pata.v05 --- ata_pata.v06 --- ata_pata.v07 --- block_cc.v00 --- ehci_ehc.v00 --- emulex_e.v00 --- weaselin.t00 --- esx_dvfi.v00 --- ipmi_ipm.v00 --- ipmi_ipm.v01 --- ipmi_ipm.v02 --- lsi_mr3.v00 --- lsi_msgp.v00 --- lsu_hp_h.v00 --- lsu_lsi_.v00 --- lsu_lsi_.v01 --- lsu_lsi_.v02 --- lsu_lsi_.v03 --- lsu_lsi_.v04 --- misc_dri.v00 --- net_e100.v00 --- net_e100.v01 --- net_enic.v00 --- net_forc.v00 --- net_mlx4.v00 --- net_mlx4.v01 --- net_vmxn.v00 --- nmlx4_co.v00 --- nmlx4_en.v00 --- nmlx4_rd.v00 --- nvme.v00 --- ohci_usb.v00 --- rste.v00 --- sata_ahc.v00 --- sata_ata.v00 --- sata_sat.v00 --- sata_sat.v01 --- sata_sat.v02 --- sata_sat.v03 --- sata_sat.v04 --- scsi_aac.v00 --- scsi_adp.v00 --- scsi_aic.v00 --- scsi_fni.v00 --- scsi_ips.v00 --- scsi_meg.v00 --- scsi_meg.v01 --- scsi_meg.v02 --- scsi_mpt.v01 --- scsi_mpt.v02 --- uhci_usb.v00 --- xhci_xhc.v00 --- tools.t00 --- xorg.v00 --- imgdb.tgz --- imgpayld.tgz
build=
updated=0

I also did a quick %s/\///g in vim to remove the first forward slash from each file.

Step 4: We need to update our PXE menu. Remember, this is the pxelinux folder inside the esx60 folder:

gsuser@tftp:[/tftpboot/esxi60]$ cat pxelinux.cfg/default
menu title Pan American Airways ESXi 6.0 PXE Menu
default menu.c32
timeout 100

label local
menu label ^A - Boot from first hard drive
com32 esxi-5.5.0.update01-1623387.x86_64/chain.c32
append hd0

label esxi-6.0.0-2494585-standard
menu label ^B - Install VMWare ESXi 6.0.0-2494585
kernel mboot.c32
append -c esxi-6.0.0-2494585-standard/boot.cfg ks=http://www2.knight139.co.uk/anaconda/kickstart/esxi-6.0.0-2494585.x86_64.cfg

MENU SEPARATOR

label esxi-6.0.0-2494585-standard_hp
menu label ^B - Install VMWare ESXi 6.0.0-2494585
kernel mboot.c32
append -c esxi-6.0.0-2494585-standard_hp/boot.cfg ks=http://www2.knight139.co.uk/anaconda/kickstart/esxi-6.0.0-2494585.x86_64.cfg
gsuser@tftp:[/tftpboot/esxi60]$

gsuser@tftp:[/tftpboot/esxi60]$ sudo umount /mnt

That’s us done now. Don’t porget to add your preseed file option to bring you up to the latest build (Sep 2015):
esxcli network firewall ruleset set -e true -r httpClient
esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-6.0.0-20150902001-standard

That’s it, you are now able to leave salting as it was previously, and have a newer build of ESXi on your menu to boot.

A Nice Game OF Chess

I’ve recently rediscovered my emulator collection, so have spent quite some time playing road Rash, Qix and Centipede Being as this is 2015, I’ve also found some Game Genie codes in true retro fashion, so I will post them here for reference:

S E G A   M A S T E R   S Y S T E M

Air Rescue
Cheat            Effect
00CC2E:07        Full Passengers
00CC1D:00        No-one to rescue

Alex Kidd In Shinobi World
Cheat            Effect
00C22E:99        Infinite Health
00C22F:FF        Infinite Lives

Arcade Smash Hits
Cheat            Effect
00C021:FF        Infinite Lives (Breakout)

Castle Of Illusion
Cheat            Effect
00C0BD:05        Infinite Health
00DC00:FF        Infinite Time
00C0C8:FF     Infinite Lives

Land Of Illusion
Cheat            Effect
00C099:05        Infinite Health

Lemmings
Cheat            Effect
00DB72:99        Infinite Climbers
00DB73:99        Infinite Umbrellas
00DB74:99        Infinite Bombers
00DB75:99        Infinite Blockers
00DB76:99        Infinite Builders
00DB77:99        Infinite Bashers
00DB78:99        Infinite Miners
00DB79:99        Infinite Diggers
00DAD3:60        Infinite Time
00DB5C:64        Always Rescue 100%

Sonic 1
Cheat            Effect
00d27f:00 \
00d2a6:00  > one of these is for all 6 emeralds
00dcf9:00 /

Sonic 2
Cheat            Effect
** NB: Disable these codes at level end/special stage!
00D299:99        Always have 99 rings

Sonic Chaos
Cheat            Effect
** NB: Disable these codes at level end/special stage!
00D29A:99        99 Rings (Every ring gives special stage/extra life)
00D2BF:99        99 Seconds (Never Run out of time)
00D2C3:FF        Infinite Continues
0A2-1EC-E6E      Each ring worth 10
3A9-DCB-2A2      Never lose rings

Cyber Shinobi
Cheat            Effect
00C012:08        Infinite Ammo
00C011:08        Infinite Ninjitsu
00c010:08        Infinite Power

Desert Speedtrap
Cheat            Effect
00CAF9:09        Ininifte Health

Marble Madness
Cheat            Effect
00DFC4:63        Infinite Time
00DFC9:01        No Kills (When the ball gets broken)

Pac-Mania
Cheat            Effect
00c040:09        Infinite Lives

Prince Of Persia
Cheat            Effect
00c292:09        Infinite Health

Special Crimminal Investigation
Cheat            Effect
00C415:99        Infinite Time
00C091:67        Infinite Speed
00C08E:09        Infinite Bazooka
00C08F:05        Infinite Turbo

Sonic 1
Cheat            Effect
00D2AA:99        Always Have 99 Rings

S E G A   M E G A   D R I V E (G E N E S I S)

Streets Of Rage
Cheat            Effect
AT4A-AA48        Infinite fighters for both players
RFAA-A6VR        Infinite special attacks for both players
AJ4T-AA34        Infinite life on life gauge both players

Back to ESX Reality

Recently, VMWare published a KB article which was picked up by El Reg which requires emergency patching of your ESX estate. This bug again relates to CBT and backups. However, if the update is not applied, VMs will also be unable to power on or indeed boot in some cases.

Here’s the quick fix I applied to correct the problem:

root@dhcp48[~]# esxcli network firewall ruleset set -e true -r httpClient
root@dhcp48[~]# esxcli software profile update -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml -p ESXi-6.0.0-20150504001-standard

Update Result
Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
Reboot Required: true
VIBs Installed: VMware_bootbank_esx-base_6.0.0-0.6.2715440
VIBs Removed: VMware_bootbank_esx-base_6.0.0-0.0.2494585
VIBs Skipped: VMWARE_bootbank_mtip32xx-native_3.8.5-1vmw.600.0.0.2494585, VMware_bootbank_ata-pata-amd_0.3.10-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-atiixp_0.4.6-4vmw.600.0.0.2494585, VMware_bootbank_ata-pata-cmd64x_0.2.5-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-hpt3x2n_0.3.4-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-pdc2027x_1.0-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-serverworks_0.4.3-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-sil680_0.4.8-3vmw.600.0.0.2494585, VMware_bootbank_ata-pata-via_0.3.3-2vmw.600.0.0.2494585, VMware_bootbank_block-cciss_3.6.14-10vmw.600.0.0.2494585, VMware_bootbank_cpu-microcode_6.0.0-0.0.2494585, VMware_bootbank_ehci-ehci-hcd_1.0-3vmw.600.0.0.2494585, VMware_bootbank_elxnet_10.2.309.6v-1vmw.600.0.0.2494585, VMware_bootbank_emulex-esx-elxnetcli_10.2.309.6v-0.0.2494585, VMware_bootbank_esx-dvfilter-generic-fastpath_6.0.0-0.0.2494585, VMware_bootbank_esx-tboot_6.0.0-0.0.2494585, VMware_bootbank_esx-xserver_6.0.0-0.0.2494585, VMware_bootbank_ima-qla4xxx_2.02.18-1vmw.600.0.0.2494585, VMware_bootbank_ipmi-ipmi-devintf_39.1-4vmw.600.0.0.2494585, VMware_bootbank_ipmi-ipmi-msghandler_39.1-4vmw.600.0.0.2494585, VMware_bootbank_ipmi-ipmi-si-drv_39.1-4vmw.600.0.0.2494585, VMware_bootbank_lpfc_10.2.309.8-2vmw.600.0.0.2494585, VMware_bootbank_lsi-mr3_6.605.08.00-6vmw.600.0.0.2494585, VMware_bootbank_lsi-msgpt3_06.255.12.00-7vmw.600.0.0.2494585, VMware_bootbank_lsu-hp-hpsa-plugin_1.0.0-1vmw.600.0.0.2494585, VMware_bootbank_lsu-lsi-lsi-mr3-plugin_1.0.0-1vmw.600.0.0.2494585, VMware_bootbank_lsu-lsi-lsi-msgpt3-plugin_1.0.0-1vmw.600.0.0.2494585, VMware_bootbank_lsu-lsi-megaraid-sas-plugin_1.0.0-1vmw.600.0.0.2494585, VMware_bootbank_lsu-lsi-mpt2sas-plugin_1.0.0-1vmw.600.0.0.2494585, VMware_bootbank_lsu-lsi-mptsas-plugin_1.0.0-1vmw.600.0.0.2494585, VMware_bootbank_misc-cnic-register_1.78.75.v60.7-1vmw.600.0.0.2494585, VMware_bootbank_misc-drivers_6.0.0-0.0.2494585, VMware_bootbank_net-bnx2_2.2.4f.v60.10-1vmw.600.0.0.2494585, VMware_bootbank_net-bnx2x_1.78.80.v60.12-1vmw.600.0.0.2494585, VMware_bootbank_net-cnic_1.78.76.v60.13-2vmw.600.0.0.2494585, VMware_bootbank_net-e1000_8.0.3.1-5vmw.600.0.0.2494585, VMware_bootbank_net-e1000e_2.5.4-6vmw.600.0.0.2494585, VMware_bootbank_net-enic_2.1.2.38-2vmw.600.0.0.2494585, VMware_bootbank_net-forcedeth_0.61-2vmw.600.0.0.2494585, VMware_bootbank_net-igb_5.0.5.1.1-5vmw.600.0.0.2494585, VMware_bootbank_net-ixgbe_3.7.13.7.14iov-20vmw.600.0.0.2494585, VMware_bootbank_net-mlx4-core_1.9.7.0-1vmw.600.0.0.2494585, VMware_bootbank_net-mlx4-en_1.9.7.0-1vmw.600.0.0.2494585, VMware_bootbank_net-nx-nic_5.0.621-5vmw.600.0.0.2494585, VMware_bootbank_net-tg3_3.131d.v60.4-1vmw.600.0.0.2494585, VMware_bootbank_net-vmxnet3_1.1.3.0-3vmw.600.0.0.2494585, VMware_bootbank_nmlx4-core_3.0.0.0-1vmw.600.0.0.2494585, VMware_bootbank_nmlx4-en_3.0.0.0-1vmw.600.0.0.2494585, VMware_bootbank_nmlx4-rdma_3.0.0.0-1vmw.600.0.0.2494585, VMware_bootbank_nvme_1.0e.0.35-1vmw.600.0.0.2494585, VMware_bootbank_ohci-usb-ohci_1.0-3vmw.600.0.0.2494585, VMware_bootbank_qlnativefc_2.0.12.0-5vmw.600.0.0.2494585, VMware_bootbank_rste_2.0.2.0088-4vmw.600.0.0.2494585, VMware_bootbank_sata-ahci_3.0-21vmw.600.0.0.2494585, VMware_bootbank_sata-ata-piix_2.12-10vmw.600.0.0.2494585, VMware_bootbank_sata-sata-nv_3.5-4vmw.600.0.0.2494585, VMware_bootbank_sata-sata-promise_2.12-3vmw.600.0.0.2494585, VMware_bootbank_sata-sata-sil24_1.1-1vmw.600.0.0.2494585, VMware_bootbank_sata-sata-sil_2.3-4vmw.600.0.0.2494585, VMware_bootbank_sata-sata-svw_2.3-3vmw.600.0.0.2494585, VMware_bootbank_scsi-aacraid_1.1.5.1-9vmw.600.0.0.2494585, VMware_bootbank_scsi-adp94xx_1.0.8.12-6vmw.600.0.0.2494585, VMware_bootbank_scsi-aic79xx_3.1-5vmw.600.0.0.2494585, VMware_bootbank_scsi-bnx2fc_1.78.78.v60.8-1vmw.600.0.0.2494585, VMware_bootbank_scsi-bnx2i_2.78.76.v60.8-1vmw.600.0.0.2494585, VMware_bootbank_scsi-fnic_1.5.0.45-3vmw.600.0.0.2494585, VMware_bootbank_scsi-hpsa_6.0.0.44-4vmw.600.0.0.2494585, VMware_bootbank_scsi-ips_7.12.05-4vmw.600.0.0.2494585, VMware_bootbank_scsi-megaraid-mbox_2.20.5.1-6vmw.600.0.0.2494585, VMware_bootbank_scsi-megaraid-sas_6.603.55.00-2vmw.600.0.0.2494585, VMware_bootbank_scsi-megaraid2_2.00.4-9vmw.600.0.0.2494585, VMware_bootbank_scsi-mpt2sas_19.00.00.00-1vmw.600.0.0.2494585, VMware_bootbank_scsi-mptsas_4.23.01.00-9vmw.600.0.0.2494585, VMware_bootbank_scsi-mptspi_4.23.01.00-9vmw.600.0.0.2494585, VMware_bootbank_scsi-qla4xxx_5.01.03.2-7vmw.600.0.0.2494585, VMware_bootbank_uhci-usb-uhci_1.0-3vmw.600.0.0.2494585, VMware_bootbank_xhci-xhci_1.0-2vmw.600.0.0.2494585, VMware_locker_tools-light_6.0.0-0.0.2494585

root@dhcp48[/vmfs/volumes/551fabc4-49da563c-9548-000c29c06bd9]# vmware -v
VMware ESXi 6.0.0 build-2494585
root@dhcp48[/vmfs/volumes/551fabc4-49da563c-9548-000c29c06bd9]# reboot
[root@dhcp48:~] vmware -v
VMware ESXi 6.0.0 build-2715440

You will also want to apply update 02 for Veem Backup and replication too, so I’d apply that.

Living In a (DHCP) Box

Recently, I had to set up a Windows only DHCP/DNS Server for a client, and so due to the limitation on Virgin Media Super Hubs, (christened the Super Fail in our office) we could not tell the built in DHCP server to pass out different DNS servers for our subnet. Virgin Media appear to have taken a perfectly functional router, and bastardised the firmware to… ooh look at the shiny!

So we deployed server core to run DHCP and DNS. So I’m going to set up a Windows DHCP Server using only Powershell, but this will also work on full server 2012 installs. I’m going to presume:

* Your DHCP Server is getting the static IP 192.168.210.4/24
* Your DHCP range is 192.168.210.100-200
* Your Primary DNS is 192.168.210.2 and secondary is 8.8.8.8
* Your default gateway is 192.168.210.1
* Your WINS Server is 192.168.210.4
* You want leases to last 15 minutes
* Your DNS Suffix is lan.local
* You want to set a fixed lease of 192.168.210.3 to Host K139ESX02 with MAC 00:0c:29:B0:0B:E5
* Your AD Server lives on 192.168.0.4 and you want to authorize DNS with it

Step 1: Launch Powershell as administrator, then run:

Get-NetIPInterface | Where-Object {$_.InterfaceAlias -eq "Ethernet0" }

This gives me InterfaceIndex 12 which we can now use to set the Static IP on that NIC:

New-NetIPAddress –InterfaceIndex 12 –IPAddress -192.0.210.4 –PrefixLength 24 –DefaultGateway -192.0.2.1
Set-DNSClientServerAddress –InterfaceIndex 12 -ServerAddresses 192.168.210.2,8.8.8.8

Step 2: Now the Network card has a static IP address, we can install DHCP:

Add-WindowsFeature DHCP

Step 3. Configuring the DHCP Server.

Well now bind DHCP to the Ethernet0 interface and add a scope for 192.168.210.0/24

Set-DhcpServerv4Binding -BindingState $true -InterfaceAlias "Ethernet0"

Add-DhcpServerv4Scope -Name "LAN" -StartRange 192.168.210.100 -EndRange 192.168.210.200 -SubnetMask 255.255.255.0 -Description "LAN" -State Inactive
Set-DhcpServerv4Scope -ScopeId 192.168.210.0 -LeaseDuration 0.00:15:00

Now we’ll add the DHCP options. Consult Microsoft’s DHCP Option Parameters at https://technet.microsoft.com/en-us/library/cc958941.aspx for more information, for clarity reasons, I’m splitting the option commands, but you can do them in one go:

Set-DhcpServerv4OptionValue -ScopeID 192.168.210.0 -DNSDomain lan.local
Set-DhcpServerv4OptionValue -ScopeID 192.168.210.0 -Router 192.168.210.1
Set-DhcpServerv4OptionValue -ScopeID 192.168.210.0 -DNSServer 192.168.210.4,8.8.8.8
Set-DhcpServerv4OptionValue -ScopeID 192.168.210.0 -WINSServer 192.168.210.4
Add-DhcpServerInDC -DNSName lan.local

Step 4: Set a fixed lease for the ESX host:

Add-DhcpServerv4Reservation -ScopeId 192.168.210.0 -IPAddress 192.168.210.3 -ClientId 00-0C-29-B0-0B-E5 -Description "ESX Hypervisor #2" -Name "K139ESX02"

Step 5: Check your working:

Get-DhcpServerv4Scope -ScopeId 10.11.210.0 | fl

ScopeId            : 192.168.210.0
Name               : LAN
Description        : LAN
SuperscopeName     :
SubnetMask         : 255.255.255.0
StartRange         : 192.168.210.100
EndRange           : 192.168.210.200
LeaseDuration      : 00:15:00
NapProfile         :
NapEnable          : False
Delay (ms)         : 0
State              : Active
Type               : DHCP
MaxBootpClients    : 4294967295
ActivationPolicies : True

Get-DhcpServerv4OptionValue -ScopeID 192.168.210.0 | fl

OptionId    : 51
Name        : Lease
Type        : DWord
Value       : {900}
VendorClass :
UserClass   :
PolicyName  :

OptionId    : 15
Name        : DNS Domain Name
Type        : String
Value       : {lan.local}
VendorClass :
UserClass   :
PolicyName  :

OptionId    : 3
Name        : Router
Type        : IPv4Address
Value       : {192.168.0.4}
VendorClass :
UserClass   :
PolicyName  :

OptionId    : 6
Name        : DNS Servers
Type        : IPv4Address
Value       : {192,168,210,4,8.8.8.8}
VendorClass :
UserClass   :
PolicyName  :

OptionId    : 44
Name        : WINS/NBNS Servers
Type        : IPv4Address
Value       : {192.168.0.4}
VendorClass :
UserClass   :
PolicyName  :

Step 6: Activating the DHCP Scope

I’ll leave it to your discretion how you move from one DHCP server to another without knocking out a network.  I did it by setting a fixed IP on a client machine with fixed DNS on 8.8.8.8. Then we can activate the Scope:

Set-DhcpServerv4Scope -ScopeId 10.11.12.0 -State Active

Then ask clients to renew their IPs. To check what the status of DHCP leases are, run:

Get-DHCPServerv4Lease -ScopeID 192.168.210.0 | Select-Object IPAddress, ClientID, AddressState

IPAddress        ClientID           AddressState
---------------  -----------------  ------------
192.168.210.3    00:0c:29:B0:0B:E5  Reserved
192.168.210.100  AA:BB:CC:DD:EE:FF  Active

And you’re done. Have a cookie!

NB: I still prefer Linux DHCP servers run ISC DHCP and BIND but there’s times when this isn’t practical