Newbie To Storage Area Networks
April 8, 2008 6:51 AM Subscribe
What's the performance difference between iSCSI and FCoE? I dunno if I'm asking the right audience here, but I figure there may be some IT pros out there who know the SAN alphabet soup.
I know that FCoE isn't really commercial yet, but I'm looking for people who might have some experience with fibre channel equipment and/or iSCSI for their Storage Area Network. Or at least point me to the right online forum where those people might hang out...?
I know that FCoE isn't really commercial yet, but I'm looking for people who might have some experience with fibre channel equipment and/or iSCSI for their Storage Area Network. Or at least point me to the right online forum where those people might hang out...?
There are software-only implementations of the entire iSCSI protocol, targets in Linux and Solaris at least and initiators for pretty much every platform so you can run an iSCSI SAN entirely on generic PC hardware. I strongly suspect your performance will suffer versus dedicated machinery but you can always add it in the future. FC requires a substantial buy-in of specialized equipment to even get started and as majick said FCoE doesn't really exist now.
posted by Skorgu at 8:48 AM on April 8, 2008
posted by Skorgu at 8:48 AM on April 8, 2008
i built a pretty affordable 11ish TB array from a company called Coraid who was pushing ATA over Ethernet. Check it out http://www.coraid.com/ (not affiliated with them in any way, yada yada yada....)
posted by joshgray at 10:01 AM on April 8, 2008
posted by joshgray at 10:01 AM on April 8, 2008
Best answer: I'm just guessing, but I'd expect FCoE to have slightly better performance. FC is designed to have a low protocol overhead, and in a typical iSCSI SAN you don't use the features of IP (like routability). So I guess iSCSI has some fat that's not really needed.
If I were a buyer I'd take iSCSI all the time, just because it's easy to understand. I work for a storage vendor, and my impression is that true FC skills are not really widespread, while everybody gets iSCSI once he's looked at a couple of traces in Wireshark.
posted by dhoe at 1:07 PM on April 8, 2008
If I were a buyer I'd take iSCSI all the time, just because it's easy to understand. I work for a storage vendor, and my impression is that true FC skills are not really widespread, while everybody gets iSCSI once he's looked at a couple of traces in Wireshark.
posted by dhoe at 1:07 PM on April 8, 2008
Best answer: fibre channel is much faster, more robust and a lot more expensive, with dedicated fibre-optic cables between the SAN drives, the fibre-channel switch, and the server(s). Usual speed is 2GB or 4GB per channel, i.e. fibre-optic cable.
iSCSI basically is SCSI commands (i.e. like a SCSI hard-drive) over TCP so can be done in hardware or software, and just routes over your existing switch network so can be fibre-optic or CAT6 1GBe. Its a bit like standard windows CIFS file sharing, but lower level and more efficient. Any server OS supports it. It's cheap and relatively simple, and routes over your standard switch network, while still have the advantage of centralised fast storage in your SAN. It is however less robust with less redundancy etc, unless you really engineer the hell out of it.
FCoE is fibre channel over ethernet, and is brand new. It's sort of a hybrid technology of the two, using fibre-channel block level commands over ethernet, instead of scsi block level commands over ethernet. It's fibre-channels way of nicking the flexibility of iSCSI, while still using fibre channel infrastructure. Frankly, unless you've a desperate need to expand an existing fibre-channel network, stick to iSCSI - FCoE is way too new to be much use yet, nothing much supports it.
If you have oodles of cash and need real speed in a local datacentre style environment, virtual servers etc etc, go for fibre channel. If you're using standard servers, especially ones spread out over an area, and are on a tight budget, go for iSCSI. You can even go for a mix, if you need to. Avoid FCoE for now, unless someone's paying you to be a case study.
That said, fibre-channel has come down a lot in price. We just got a 1TB fibre-channel EMC SAN to store all our VM server images on for £8000 (plus a bunch more for the ESX servers themselves, but heyho). A big chunk of that is the 10K SAS drives, but it's to replace our entire existing server system, and then some.
posted by ArkhanJG at 4:02 PM on April 8, 2008 [1 favorite]
iSCSI basically is SCSI commands (i.e. like a SCSI hard-drive) over TCP so can be done in hardware or software, and just routes over your existing switch network so can be fibre-optic or CAT6 1GBe. Its a bit like standard windows CIFS file sharing, but lower level and more efficient. Any server OS supports it. It's cheap and relatively simple, and routes over your standard switch network, while still have the advantage of centralised fast storage in your SAN. It is however less robust with less redundancy etc, unless you really engineer the hell out of it.
FCoE is fibre channel over ethernet, and is brand new. It's sort of a hybrid technology of the two, using fibre-channel block level commands over ethernet, instead of scsi block level commands over ethernet. It's fibre-channels way of nicking the flexibility of iSCSI, while still using fibre channel infrastructure. Frankly, unless you've a desperate need to expand an existing fibre-channel network, stick to iSCSI - FCoE is way too new to be much use yet, nothing much supports it.
If you have oodles of cash and need real speed in a local datacentre style environment, virtual servers etc etc, go for fibre channel. If you're using standard servers, especially ones spread out over an area, and are on a tight budget, go for iSCSI. You can even go for a mix, if you need to. Avoid FCoE for now, unless someone's paying you to be a case study.
That said, fibre-channel has come down a lot in price. We just got a 1TB fibre-channel EMC SAN to store all our VM server images on for £8000 (plus a bunch more for the ESX servers themselves, but heyho). A big chunk of that is the 10K SAS drives, but it's to replace our entire existing server system, and then some.
posted by ArkhanJG at 4:02 PM on April 8, 2008 [1 favorite]
"That said, fibre-channel has come down a lot in price. We just got a 1TB fibre-channel EMC SAN to store all our VM server images on for £8000..."
That's definitely a downward trend for FC gear as price goes, but that $16,000 you just spent would pay for the iSCSI chassis I just bought last month, a couple TB of disk, and left some over to buy an extra ESX box, a few iSCSI HBAs (The qlogic 4060c can be had quite cheaply) or a VirtualCenter license.
The price differential between iSCSI and FC is -- even with FC storage vendors' desperation mounting rapidly as they realize they've completely and utterly priced themselves out of the small enterprise -- still quite significant, even considering the feature gap. Also, storage vendors are starting to realize that forcing buyers to pay 200-400% markup on commodity disks isn't going to last forever, and prices are falling across the board regardless of the interconnect being used on the SAN. FC is getting cheaper (or rather, "less ridiculously overpriced") but so is iSCSI.
posted by majick at 4:27 PM on April 28, 2008
That's definitely a downward trend for FC gear as price goes, but that $16,000 you just spent would pay for the iSCSI chassis I just bought last month, a couple TB of disk, and left some over to buy an extra ESX box, a few iSCSI HBAs (The qlogic 4060c can be had quite cheaply) or a VirtualCenter license.
The price differential between iSCSI and FC is -- even with FC storage vendors' desperation mounting rapidly as they realize they've completely and utterly priced themselves out of the small enterprise -- still quite significant, even considering the feature gap. Also, storage vendors are starting to realize that forcing buyers to pay 200-400% markup on commodity disks isn't going to last forever, and prices are falling across the board regardless of the interconnect being used on the SAN. FC is getting cheaper (or rather, "less ridiculously overpriced") but so is iSCSI.
posted by majick at 4:27 PM on April 28, 2008
This thread is closed to new comments.
"people who might have some experience with fibre channel equipment"
FC gear as it exists today is not FCoE, and it's a different beast. For one thing, it exists, and for another it requires expensive fiber switching equipment. It has very different performance characteristics from 1GE iSCSI since bandwidth is higher and latency is lower. Cost per terabyte is also quite a bit higher as well, so if one of your performance metrics is "bang for the buck," an FC SAN isn't necessarily the first choice.
For what it's worth, when most recently faced with the choice of SAN infrastructure, iSCSI was my choice by a mile.
posted by majick at 7:31 AM on April 8, 2008