I’ve spent the last week or so doing some research to figure out what I want to achieve and have built up a hefty list of requirements. By the time I’ve finished this I want to have the following in my home lab:
- Several VMWare ESXi hosts supporting all commercial features for failover
- Initially just run ESXi 5.5 on a free licence before purchasing a commercial use one
- Ideally locate secondary hosts off-site for failover purposes
- Support for Mac OS X VMs in addition to all other operating systems
- Very low noise environment
- Full VOIP connectivity throughout my home office areas
- Keep everything contained in a rack (ideally half height)
- Initially use virtual routers before moving over to Cisco kit
To assist with this I’ve gone on a research and purchasing spree. I wanted to select hardware that would allow me to run hardware passthrough to VM hosts if I desired, wanted all my systems to run without contention on the hardware and power and noise are primary concerns.
I’ve gone for the following configuration:
- 2x AMD Opteron 4334 6 core 3.1GHz CPUs
- Asus KCMA-D8 motherboard with remote management KVM add-on
- Intel 320 series 40GB SSD to install ESXIs on
- WD 500GB black temporary drive for VMs until I get my SAN setup
- 4U XCase
- 550W low-noise PSU (we’re not powering many drives here)
- Noctura NH-U12D0 heatsink and 12cm fans
- 32GB RAM
All together this has come in at around £900 including VAT and shipping with parts from eBay, Amazon and a few bit ordered from the states. I’d made my mind up on the motherboard and CPU coolers but everything else has been based on price.
The AMD chips had a big impact in price. I must admit I felt a bit reserved. When I was young my pocket money only ever allowed me to buy AMD chips and when I finally got my first Intel chip I never looked back. That being said when I worked at HP in 2005 I remember we got one of the first HP Opteron servers in. It was a beast for that time and absolutely kicked the ass off the Intel servers we had in our lab that were about the same cost. One of my early requirements was the IOMMU support to pass through hardware to hosts which AMD meet so I thought to save a few pounds it’s worth having a look how good the current AMD chips are.
The Noctura fans are giant. I’m actually slightly concerned whether they’ll even fit in a 4U case after all mounts. They’re also not cheap (about the same cost as the CPU). I’ve seen exceptional reviews for them though and don’t want to skimp on cooling or noise components.
The other big decision was for the ASUS KCMA-D8. I wanted a multi-socket solution to get the more cores for my money. I also wanted to control the budget a little so ruled out some of the more expensive Supermicro options. The reviews of this board seem pretty solid and it’s been used by a couple of other people in ESX labs. The last time I used an Asus board it cost me about £20 new and blew up when I added a USB scanner (USB was hardly used at the time). I’m hoping they’ve come on a little way in the last 15 years.
Incidentally I decided to go for the SSD rather than just using a memory stick for the ESX installation as it was dirt cheap (only £20 on Amazon) and also gave me enough additional storage that I could put several ISOs on there. Something still feel wrongs using a flimsy plastic memory stick to hold the OS at the core of all your infrastructure.
I’m quite happy with £900 for a practically silent 12 core 32-GB RAM system. The client servers that inspired this project run a rather hefty facility on 20 cores but come in at about ten times the price to be packed into 1U by Dell.
I’ve also ordered a couple of parts for my network to support this. My house has an office above the garage (separate from the rest of the house). The office is cabled via 2x CAT6 cables to the main house that already has a patch panel underneath the stairs with Ethernet in every room. My office will be my “data centre” whilst I’ll have a slightly updated wiring closet under the stairs.
My only real requirement for switches were that they supported VLANs, were reliable, gigabit and practically silent.
I’ve purchased two HP 1810-24G switches and a Draytek Vigor 120 ADSL Ethernet modem (exposes ADSL as PPPoE to consume with a virtual router easily). These came in at just over £300.
I’m waiting on a couple more parts to turn up but will post some pictures of my initial build and configuration shortly.