Developer seeks big memory, many cores

November 20, 2012 12:19 PM

Looking for advice on a computer to buy or build that will have a ridiculous amount of RAM and many processor cores.

I'm a programmer and do a lot of development in virtual machines. I want to be able to run a lot of them concurrently on my desktop computer, though the raw performance of each of them isn't all that important. I'm looking for advice on the system that will offer the best value for doing that.

A few other salient points:

• The host operating system will be Linux, most likely CentOS or Ubuntu. I don't plan on running much on the bare metal, but I want to avoid proprietary or obscure hardware that might give me trouble.
• Each VM will be allocated 2-6 GB of RAM, and I want to run as many of these as I can — certainly at least five, and more would be better. I'll need the computer to have at least 32GB of RAM, but 64 GB (or maybe support 64 GB for a future upgrade) is preferable.
• I want enough processor cores to run all the VMs and still have the host be responsive for interactive GUI work.
• It's a desktop machine, so less noise and heat is a plus
• I'm not a gamer, so I don't want to pay for fancy case lights or high-end video cards. I do need gigabit Ethernet, but don't need WiFi, nor do I need a Bluray drive, monitor, mouse or keyboard.
• I'm capable of building a system from scratch, but my knowledge of the desktop hardware market is a little outdated.
I don't really have a specific budget for this, except that less expensive is better. I normally buy machines off Craigslist or loss leaders from big box stores, but can't find anything now that's close to what I'm looking for. I'd appreciate any advice you may have.

posted by tomwheeler to Computers & Internet (15 answers total)

How about EC2? That way, you're not buying and maintaining expensive hardware, and you can expand your computing power as needed.
posted by Blazecock Pileon at 12:32 PM on November 20, 2012 [2 favorites]

There are a lot of options other than having hardware on your premises. The aforementioned EC2 is one. You can also look at Slicehost or Linode for more traditional VMs. Oh, hell, just abstract away everything and look at EngineYard or Heroku. Perhaps you need to tell us why you want to own hardware?
posted by chengjih at 12:39 PM on November 20, 2012

You're not going to beat the cloud computing folks for this particular workload. EC2 is great. Azure works well, especially for services running on Windows (which you're not.) Rackspace is all right, a bit more fiddly than Amazon but could work well if you know your way around provisioning a machine. There's a cool startup here in Seattle called Skytap that's all about fast provisioning of throwaway machines.

At this point it's a suckers game to run "big iron" machines at home. You don't have the advantage of buying at scale (multiple 1000s of machines).

That said, if you're still looking to do this, I'd camp out at eBay on the "blade server" search. Picking up several older machines from ~3 years ago at $300-500 dollars will net you much, much better price/performance than any newly constructed machine.
posted by Exonym at 12:41 PM on November 20, 2012

Putting that much capacity into a single server is not going to be cheap - you're a long way from commodity hardware.

Aside from EC2, if you wanted to keep everything local you could just buy a bunch of cheap desktop machines - you can easily find them for $300-500 each, and then build a local gigabit network.

Otherwise a quick check of newegg.com shows a 64-bit 3.somethingGHz machine w/16GB RAM (upgradeable to 64GB) is $1200. Add some memory and you're done? Then buy a silent power supply to make it quieter.
posted by jpeacock at 12:43 PM on November 20, 2012

Best answer: Checkout Lenovo S30 Workstations.
Xeon processor, ECC memory upgradeable to 64GB (UDIMM) or 128GB (RDIMM)
posted by Consult The Oracle at 12:48 PM on November 20, 2012

Best answer: This will definitely kill any budget, but the Dell R7XX or R9XX series are pretty much what you'd want to look at. The R710 is dual processor, the R910 is quad core. The r710 is now no longer being sold by Dell, but they're up on ebay; you can get them around $2k-ish mostly bare, and then toss in up to 18 16GB RDIMMS for 288GB of ram. The r910 can take up to 2TB of RAM.
posted by nobeagle at 12:50 PM on November 20, 2012

I n^th the point that the modern way to do this is by cluster -- 10 cheapo machines will be much better than one uberbox. You can either buy them yourselves and work out the clustering, or pay someone to do it for you (in the cloud).
posted by katrielalex at 12:54 PM on November 20, 2012

Ah, sorry I missed the part about it being a desk top machine; the dell systems I pointed you at are definitely not desktop machines. But if this is for work, and you have a legitimate need of something local, you might want to just stick it somewhere that the noise/size won't be an issue.

I'd say that 10 cheapo machines might be cheaper up front, but will cost more in heat and power than an uber box, and admining 10 machines, which might not even be all the same hardware is way more work than admining one xen box.
posted by nobeagle at 1:00 PM on November 20, 2012

Best answer: I'm assuming that you're being honest with yourself and what you listed is what you really need. It does seem a bit extreme. Most of the time you don't need to run all the VM's all at once -- in my experience, I've been able to run about the configuration you describe split between an 8G 4 core desktop and an 8G 4 core laptop. I generally just "park" (save state) the VM's I'm not using.

Given the size and number of VMs you want, EC2 or similar aren't really going to be cost effective. You don't seem to be looking for big iron or externally reachable network connectivity. . You can buy two machines capable of running roughly your workload (16G, 4/8 cores, no gfx, use remote desktop or ssh) for about what you would rent a two VMs for over a year (one of your vm's is $175/mo at rackspace).

So, like someone else suggested above -- I would consider multiple machines. You can always use a remote desktop solution to have them appear as one machine. I do this.

This will be starting to more closely approximate a server rack, of course -- but it will be substantially cheaper in the long run and the maintenance issue really is overblown if it's just for a single dev.
posted by smidgen at 1:08 PM on November 20, 2012

Best answer: You don't need a core per VM, just buy a modern single processor desktop mainboard and CPU combo:

AMD

32GB maximum, 8 cores - Add a large Samsung 840 SSD, and 32GB of approved memory and an 850watt PSU + a case, done.

If you really like intel, get an intel based combo, similar, you really don't need a core per VM and you really don't need to do multiple VCPU's, it's a waste unless your machines support it. Also disable hyperthreading, it's pointless in VMware outside of some edge case uses.

So, buy a modern desktop board, max out the memory, get an "ok" cpu for it with at least 4 cores, and be done.

IO is going to be your biggest problem and limiter anyway, so maybe a couple of SSD's, one for system, one for VM's.
posted by iamabot at 1:11 PM on November 20, 2012 [1 favorite]

Stay away from EC2/etc, it's not worth it, you're better off with buying slices.
posted by iamabot at 1:12 PM on November 20, 2012

Response by poster: Thanks for the advice, everyone. I know about EC2 and really do want to do this with local VMs. I'll constantly be building new ones for the kind of work I need to do, plus using relatively large amounts of data, so bandwidth costs would add up quickly. And I really do need to run several of them at once, because I'm working software for distributed computing. Some of the replies give me hope that I can buy or build what I need pretty inexpensively.
posted by tomwheeler at 1:22 PM on November 20, 2012

The second half of Andrew Connell's description of his non-portable development rig would seem to fit your description. He only has 48GBs of RAM but this article was written 18 months ago so likely you could get more memory and perhaps slightly better overall specs while still spending less money than he did. Granted he is running Windows but I can't see why this advice wouldn't be applicable to Linux.
posted by mmascolino at 7:43 PM on November 20, 2012 [1 favorite]

I do development in VMs on my personal machine, and I'm running an i7 w/16gb ram. By far the biggest bottleneck is hard drive speed. Get the ram you need, and then spend your money on multiple SSD hard drives. Spread the VMs out across the hard drives, and you'll be much happier.
posted by blue_beetle at 7:43 PM on November 20, 2012 [2 favorites]

Response by poster: As a postscript to my question, I was able to take advantage of some online Black Friday deals and find the components needed to build an eight-core machine with 32GB of RAM and a couple of 3TB SATA disks, all for about $800.
posted by tomwheeler at 4:50 PM on November 25, 2012