infrastructure

i run three hypervisors. two of these run 24/7 and are based on SFF dell R710s and the third is an LFF dell r710. the hypervisors themselves run VMWare ESXi with licenses from VMUG. backups are done via restic and backed up to my business gsuite account.

rack

i use a startech 25u rack. there’s nothing inherently special about it, everything is balanced over double-failsafe surge protectors. i have a mini kvm switch i can use to swap keyboard/mouse/monitor between machines, velcro straps to keep cables tidy and all servers are full sliding-racked with bezels.

esxi1

dual L5640 Intel CPUs and 48gb ECC ram, coupled with 8x500gb western digital 2.5inch hard drives in raid6. this gaves ~4tb of usable space via an H700 with the redundancy for 2 drives to fail without dataloss.

esxi2

identical components as esxi1. this was initially a dev box where I’d frequently spin up and term VM’s while testing new things. as the lab grew I’ve had to lean on it more and it now runs 24/7.

esxi3

dual L5640 Intel CPUs and 24gb ECC ram. storage here is 4x500gb SSDs in raid10 again via an H700. the primary use for this is for eventually hosting two plex servers where ram isn’t as much of a concern. it also hosts vcenter and currently, is only online to periodically update plex or move VMs between hypervisors.

opnsense

Intel X3470 CPU with 8gb ECC ram and storage via 2x 140gb SSDs in raid1.

virtual machines

i pretty much solely use ubuntu for everything. that’s on the 30-40 machines at my lab and the ~10 machines i rent at various providers. when provisioning new servers at nuked.me i also use ubuntus most recent long term release.
i use jenkins and ansible to build upon new machines and maintain/update them daily.

other

i have a cache of backup CPUs and about ~72gb ram, along with spare H700s, spare drives and PSU’s in case of failure which will someday come in handy.

administration is done via either LAN access & dell iDracs or via VPN & SSH. anything web-based (but locked to lan, such as ESXi and IDracs) are secured behind SSL certs backed by my own certificate authority setup within opnsense and locked-down nginx rules. all SSH is limited to LAN access only with key-based authentication and strict crowdsec rules.