daddy, i want more io
March 1, 2011 10:12 AM Subscribe
How to maximize IO for a fairly large dataset to a single machine?
posted by H. Roark to computers & internet (8 answers total) 3 users marked this as a favorite
We have some seriously IO bound processes, and a huge imbalance between cores and IO capacity. Currently we are using NFS via netapp, to approx 10 machines with each machine having 24 fast cores. Each core can realistically process about 100MB/sec of incoming IO, and currently our netapp is driving an aggregate of about 800MB/sec, or ~80MB/sec/node, meaning we are using about 5% of the CPU available.
The size of the dataset is around 40TB total.
What are some good ways to maximize IO and minimize total processing time? Im leaning towards direct connect FC/pci on a single machine and trying to saturate a machine instead of NFS, but outside of big SAN investments I dont see 40TB fitting on pci/ssd drives.
The RAMSAN / TMS systems are nifty, but at $30/gig they are outrageously priced.