Attention VMware experts: Seeking 2nd opinion on this proposed build

June 14, 2016 1:13 PM

I put together a proposed build for a white box ESXi 6.0 host. The last time I did something like this, I found out (too late) that the on-board SATA controller was incompatible with ESXi. So I’d like someone to verify that I’m not doing something stupid this time.

I already have three ESXi hosts running under a VMware Essentials license. This new build is slated to replace the least powerful of the three. My go-to VMware guy is gone, so I’m left to puzzle this out myself, using my Google-fu and whatever advice I can get from AskMeFi.

My build is based (with some modifications) on the build described here.

Here’s what I have:

ASRock X99 Extreme4 motherboard
Intel i7-5820K processor
Samsung 16GB DDR4 RAM x 4
Samsung EVO 250GB SSD for booting the OS (ESXi 6.0)
Western Digital Red 4TB NAS hard drive x2 for the VMs
Intel PRO/1000 dual-port gigabit NIC x 2

I’m not sure about using RAID. I don’t think that ESXi supports the on-board RAID, so I was just going to use the drives as standard SATA drives.

Does this sound reasonable? Any other advice?

Thanks!

posted by alex1965 to Computers & Internet (13 answers total)

I'm sorry not to answer your question directly, but suggest you post in the VMware community forums which I've found immensely helpful in the past.
posted by anadem at 1:40 PM on June 14, 2016

Just to throw this one out there, but my coworkers who do a lot more vmware stuff than me* just... Find compatible HP/Dell/lenovo cheapo servers and buy those.

Will they come with 64gb of ram? No, but it's pretty easy to look up "dell 110" on the vmware forums or a compatibility list and know it'll be fine.

The last time i needed to order a sort-of-decent server, dell had certain models from 6 months or a year ago on clearance for so damn cheap i couldn't have even bought the parts used for that price and they were pre-listed as compatible with the apps i needed to run, vetted from the vendors.

You can save a few bucks and know it's going to work, is what i'm saying. Roll your own is for gaming machines or really specialized stuff as far as i'm concerned(and even then i usually at least explore upgrading some off the shelf box)

*(i've used it quite a bit, but i'm more of "a guy who can win at racing games and drive a manual", and they're more of amateur/local stage rally drivers who wrench)
posted by emptythought at 2:03 PM on June 14, 2016 [1 favorite]

I recently built about the same machine, apart from I went for an ASUS Z170-K and a 6700K and I used the Intel Ethernet Server Adapter I350-T2.

I was secretly hoping the onboard NIC would work, but I needed the Intel NIC to play nicely (the installation wouldn't continue until it was happy with the hardware). Oh, and I had to flip on the CPU Hardware Virtualization support on in the BIOS, easy stuff-- and the ASUS bios is certainly nicer then the Dell ones!

Installation was a breeze, no complaints at all— I intend to just throw in SSD's for each VM as required. If all is good, I'm going to build an identical one to act as a standby-- the Plus license for the vMotion/HA stuff is a bit sharp for me right now, but I've got Acronis providing me with the poor mans version of that that meets our needs just fine.
posted by Static Vagabond at 2:36 PM on June 14, 2016

Best answer: Getting someone else's prebuilt system is generally a good idea. However, if you want to build this yourself...

Generally speaking, what you've built is a workstation, but you might have better success with a server-grade motherboard such as the Supermicro X10SRL and a Xeon E5-16xx v3 based system. The X10SRL will be a little more than the ASRock workstation board, but will come with Intel ethernets. It won't have a handful of useless PCIe x16 slots on it, which would generally be pointless on a headless hypervisor, a total waste of PCIe lanes. The X10SRL will have more PCIe slots of a server-sized x8 or x4 (but does have two x8-in-x16 slots in case you need a GPU). The i7 doesn't support the ECC memory you've selected, but the Supermicro and a Xeon E5 will. The Xeon CPU is probably more expensive. However, you'll end up with a system that vSphere can control via IPMI/BMC, and will have ECC, and is a standard, well designed platform on which LOTS of sites run ESXi.

ESXi doesn't support software RAID, which includes any chipset-based RAID that might be offered by the ASRock board. If you're interested in redundancy for your datastores, the least expensive option is to try to find a supported used RAID controller on eBay. The LSI 9240 based ones lack a write cache and will tend to be very slow (think 15MB/sec writes for HDD) but there are some LSI 9260 ones that can be found with the BBU or supercap options that are very reasonable.
posted by jgreco at 3:59 PM on June 14, 2016 [1 favorite]

First, don't waste that Samsung EVO as the ESXi boot drive. All ESXi needs is a USB memory stick to boot from. Once booted, ESXi writes very little back to the boot volume. My home VMware server has been booting off a cheapo 1GB USB stick for 4 years. Just move scratch to persistent storage. Your SATA disks will be your bottleneck so maybe you can use the SSD as flash cache. Essentials might not be licensed for that though.

The two hardware components that are tricky to source cheaply with a home ESXi build are the NICs and the RAID cards. I don't bother with RAID at home as I do backups. For NICs I found $10 TP-Link Gb cards on Amazon that work. For RAID, I've read most people pick up used HP or IBM RAID cards off ebay.

The rest of your build looks great compared to my Core 2 Quad server.
posted by LoveHam at 5:18 PM on June 14, 2016

Personally for the money and compatibility aggravation, I'd be more likely to pick up a used Dell r710.
posted by fings at 5:58 PM on June 14, 2016 [1 favorite]

Other friendly places to ask or research:

• reddit.com/r/vmware or /r/homelab
• Open Homelab Project (well, more of a wiki but you can poke around)
• a search for "ESXi white box 2016" works pretty well for builds and guides
• ask on Twitter using the hashtag #vexpert and you'll no doubt get some eager help (although the advice here was already good!)

posted by troyer at 6:00 PM on June 14, 2016

I assumed your white box build was for a home server initially. If this is intended for a professional environment, don't build a white box.
posted by LoveHam at 4:36 AM on June 15, 2016

Response by poster: Thanks for the answers, particularly jgreco's advice to consider using a server motherboard. This is the current build I'm exploring:

Supermicro X10SRA-F ATX LGA2011-3 Motherboard
Intel Xeon E5-2609 V4 1.7GHz 8-Core Processor
Kingston 64GB (4 x 16GB) Registered DDR4-2133 Memory
Samsung 850 EVO-Series 250GB 2.5" Solid State Drive
Western Digital Red 4TB 3.5" 5400RPM Internal Hard Drive x 2
Intel E1G42ETBLK PCI-Express x4 10/100/1000 Mbps Network Adapter

I'm awaiting a price quote from a local systems integrator to see if this is within my budget.

Of course, any further feedback would be much appreciated.
posted by alex1965 at 6:43 AM on June 15, 2016

Response by poster: Oh, a number of people suggested buying a pre-built system. I'm doing the white-box route because a local computer store will build it at-cost for us (we're a non-profit organization, and the owners like us).
posted by alex1965 at 7:04 AM on June 15, 2016 [1 favorite]

Best answer: The X10SRA-F is still basically a workstation motherboard, though it will almost certainly not give you any issues.

The E5-2609 v4 isn't a fantastic processor, but your hypervisor isn't particularly large, so that's probably fine. I usually guide uniprocessor configurations away from the E5-26xx unless they're going to the high end. I'd suggest an E5-1620 v3 instead of an e5-2609 v3 for example, but the addition of the two cores to the 2609 v4 changes that to a wash.
posted by jgreco at 12:40 PM on June 15, 2016

Response by poster: OK, so here's my latest proposed build:

Supermicro MBD-X10SRL-F motherboard
Intel Xeon E5-1620 v3 Haswell-EP 3.5 GHz 4 x 256KB L2 Cache 10MB
Crucial 64GB (4 x 16GB) 288-Pin DDR4 SDRAM ECC DDR4
SUPERMICRO SNK-P0050AP4 Heatsink
SAMSUNG 850 EVO 2.5" 250GB
Western Digital Red 4TB NAS Hard Disk Drive x 2
Fractal Design Define R5 Black Silent ATX Midtower Computer Case
SeaSonic Platinum SS-860XP2 860W ATX12V / EPS12V power supply
posted by alex1965 at 4:29 PM on June 15, 2016

Response by poster: I ended up ordering the build above, but with two small modifications (lower-wattage power supply, and faster [7200 RPM] Western Digital Re 4TB Datacenter hard drives).

By the way, I did look into getting a Dell server, but I couldn't find anything (refurbished or otherwise) that came even close to the specs of the system I ultimately ordered.

Thanks go out to everyone for advice, particularly jgreco.
posted by alex1965 at 3:34 AM on June 29, 2016 [1 favorite]