How to provide power to a dozen (or more!) home computers?
July 11, 2009 10:21 AM Subscribe
My sister and brother-in-law have a large (12) and growing number of desktop boxes at home. They're rightly concerned that they're putting a strain on the electrical circuits they're using. What should they be asking an electrician to do to set them up correctly?
No problems with networking or internet service here, just power. It sounds from her description (note that I'm in Seattle and they're in Chicago) that the main electrical panel is relatively new and sized appropriately. The computers are in a finished basement room.
They're not sure what to ask for & would like to be somewhat informed before talking to an electrician. With this level of power demand (with some room for growth) would they be okay with new dedicated circuits run form the main panel, a new subpanel in the basement with dedicated circuits, might they need a new main panel, or what?
Cost is not a primary issue - this will be a business expense.
Secondary question: any recommendations for an electrician to handle this type of thing in suburban Chicago (Glen Ellyn)?
Tertiary question: They're using their computers for raw computational power, nothing internet-related. If I was hip & in the know, what would I call the individual computer? I suspect "server box" is not appropriate.
No problems with networking or internet service here, just power. It sounds from her description (note that I'm in Seattle and they're in Chicago) that the main electrical panel is relatively new and sized appropriately. The computers are in a finished basement room.
They're not sure what to ask for & would like to be somewhat informed before talking to an electrician. With this level of power demand (with some room for growth) would they be okay with new dedicated circuits run form the main panel, a new subpanel in the basement with dedicated circuits, might they need a new main panel, or what?
Cost is not a primary issue - this will be a business expense.
Secondary question: any recommendations for an electrician to handle this type of thing in suburban Chicago (Glen Ellyn)?
Tertiary question: They're using their computers for raw computational power, nothing internet-related. If I was hip & in the know, what would I call the individual computer? I suspect "server box" is not appropriate.
Best answer: It really would help to know how much wattage (and thus current) these machines are pulling. If they are purely using the CPU to do calculations, most CPUs range from 65-125W these days and thus a computer using them would then pull from 150W-250W (just a rough estimate) If they're using graphics processors (GPUs) to do things like Folding@home and such, this number would be anywhere from 350-800W with dual GPUs.
If it's the first case, (150 to 250W)*12 machines/110V= 16.3A to 27.3A draw so they'd need to be on more than one circuit if their breakers are a standard 15 or 20A
If it's the second case, (350 to 800W)*12/110V= 38A to 87A which is serious business and they might need to look into rewiring
I may have made some sort of mistake in these calculations but it's a rough estimate. There's probably other factors I didn't consider though.
posted by JauntyFedora at 10:47 AM on July 11, 2009
If it's the first case, (150 to 250W)*12 machines/110V= 16.3A to 27.3A draw so they'd need to be on more than one circuit if their breakers are a standard 15 or 20A
If it's the second case, (350 to 800W)*12/110V= 38A to 87A which is serious business and they might need to look into rewiring
I may have made some sort of mistake in these calculations but it's a rough estimate. There's probably other factors I didn't consider though.
posted by JauntyFedora at 10:47 AM on July 11, 2009
Best answer: This is reasonably easy to figure out, basically a typical home has a single 110v @ 20amp or 15amp circuit run to a room giving you about 1500 or 2000 watts to play around with which you're probably right up against the barrier of where you're getting close to straining the breakers and probably the circuits running to the room, what I would recommend is that you get two dedicated 220V, 20A or 30A circuits run to the room and buy two pretty good sized APC UPS units and use these exclusively for powering the machines in the room. All modern computers have auto switching power supplies and can accept 110 or 220 power.
You're probably best off buying a standard 42u rack and installing it in the room and putting all the systems (and future ones) in the rack, and you probably also want to consider a mechanism for venting the waste heat from the room.
Your typical dell or hp single RU server has 800w power supplies and uses 600w of power at load, assuming you're not putting redundant power supplies in the machines, so figure with 220v power consuming about 3A per server, at 110v power you're going to be burning about 5A per server at load and start up.
The engineering on power is pretty straight forward, but to be honest you are going to cross a threshold where you're just not going to be able to get rid of the waste heat. If you're going for direct numbers crunching and you need to keep the machines in a home then invest in a closed cabinet, battery backups to the machines can go down gracefully and a solid mechanism for keeping it all cool and out of the general workspace.
Most of my experience is in engineering power for datacenters, but if they are all desktop users the rules is pretty much the same and big dedicated circuits with power strips off of them is still the way to go.
posted by iamabot at 10:51 AM on July 11, 2009
You're probably best off buying a standard 42u rack and installing it in the room and putting all the systems (and future ones) in the rack, and you probably also want to consider a mechanism for venting the waste heat from the room.
Your typical dell or hp single RU server has 800w power supplies and uses 600w of power at load, assuming you're not putting redundant power supplies in the machines, so figure with 220v power consuming about 3A per server, at 110v power you're going to be burning about 5A per server at load and start up.
The engineering on power is pretty straight forward, but to be honest you are going to cross a threshold where you're just not going to be able to get rid of the waste heat. If you're going for direct numbers crunching and you need to keep the machines in a home then invest in a closed cabinet, battery backups to the machines can go down gracefully and a solid mechanism for keeping it all cool and out of the general workspace.
Most of my experience is in engineering power for datacenters, but if they are all desktop users the rules is pretty much the same and big dedicated circuits with power strips off of them is still the way to go.
posted by iamabot at 10:51 AM on July 11, 2009
Best answer: At peak load you probably shouldn't have more than 3 PCs on a given circuit. Maybe 5 if they're just servers and aren't drawing extra current for DVD drives and monitors and such. But even so, that's a lot of PCs.
A new main panel would work. A subpanel might work assuming there's space in the main panel from them to run a circuit from. A subpanel would probably make the final runs easier as it'll be closer. It will seem kind of weird when they sell the house and one room is wired for a grow-op but whatever.
Anyway, just pick a number to ballpark for power draw. A desktop PSU is rated around 300-600W but draws more like 150W under light load. So assuming they actually use these systems let's say it's 300W peak. So 12 machines times 300 W is 3600W which is about 30 A so it's at least two household circuits (15A each) but you'd be better off with 3 at least in case peak draw is higher (are they rendering 3D movies?). A real electrician can give a better assessment than I can whether this is better served with 2-3 regular circuits or a subpanel.
An extra 30A draw may put you over what your main panel is rated for (typically 100A) but again you'll need a real electrician to make that call.
Finally, if they have generic desktops then they're probably not getting a good ratio of computing power per watt. They might want to invest in a server-grade machine (but maybe they already have that). The cost of the machine might end up paying for itself in lowered electricity consumption. If they're really just using raw CPU cycles then they should invest in a box with a lot of cores - like a box that has 2 sockets and a couple quad-core processors. That will give a much better bang per watt. Servers with more than 2 sockets start to get disproportionately expensive. 8 cores seems to be the current sweet spot.
This is a real problem that large-scale datacenters face and that's what they've generally been doing - moving to much larger servers that reduce the average watts per cpu core. In their case it's not a cost issue as much as that they're hitting hard limits on the amount of power they can deliver to the datacenter, much like your situation.
posted by GuyZero at 10:56 AM on July 11, 2009
A new main panel would work. A subpanel might work assuming there's space in the main panel from them to run a circuit from. A subpanel would probably make the final runs easier as it'll be closer. It will seem kind of weird when they sell the house and one room is wired for a grow-op but whatever.
Anyway, just pick a number to ballpark for power draw. A desktop PSU is rated around 300-600W but draws more like 150W under light load. So assuming they actually use these systems let's say it's 300W peak. So 12 machines times 300 W is 3600W which is about 30 A so it's at least two household circuits (15A each) but you'd be better off with 3 at least in case peak draw is higher (are they rendering 3D movies?). A real electrician can give a better assessment than I can whether this is better served with 2-3 regular circuits or a subpanel.
An extra 30A draw may put you over what your main panel is rated for (typically 100A) but again you'll need a real electrician to make that call.
Finally, if they have generic desktops then they're probably not getting a good ratio of computing power per watt. They might want to invest in a server-grade machine (but maybe they already have that). The cost of the machine might end up paying for itself in lowered electricity consumption. If they're really just using raw CPU cycles then they should invest in a box with a lot of cores - like a box that has 2 sockets and a couple quad-core processors. That will give a much better bang per watt. Servers with more than 2 sockets start to get disproportionately expensive. 8 cores seems to be the current sweet spot.
This is a real problem that large-scale datacenters face and that's what they've generally been doing - moving to much larger servers that reduce the average watts per cpu core. In their case it's not a cost issue as much as that they're hitting hard limits on the amount of power they can deliver to the datacenter, much like your situation.
posted by GuyZero at 10:56 AM on July 11, 2009
Best answer: It would depend on how far away the main panel is from the basement. If it is close by, do individual runs. It would be hard to justify a sub-panel when the main panel is ten feet away.
Now, I personally would go with a sub-panel as you can more easily add an automatic generator panel in the future. Of course, that would be with a good UPS too.
Either way, the outlets should be set up with isolated grounds. That being, each ground is only connected to that one outlet, ran back to the panel, and connected to the ground strip in the box. Then use an isolated outlet. They are special outlets that do not have any of the metal components connected to the ground lug. Just the ground plug itself.
That's my 2ยข's. I'm sure I will get change back!
posted by Climber at 10:58 AM on July 11, 2009
Now, I personally would go with a sub-panel as you can more easily add an automatic generator panel in the future. Of course, that would be with a good UPS too.
Either way, the outlets should be set up with isolated grounds. That being, each ground is only connected to that one outlet, ran back to the panel, and connected to the ground strip in the box. Then use an isolated outlet. They are special outlets that do not have any of the metal components connected to the ground lug. Just the ground plug itself.
That's my 2ยข's. I'm sure I will get change back!
posted by Climber at 10:58 AM on July 11, 2009
Best answer: two dedicated 220V, 20A or 30A circuits run to the room
This is actually really good advice, but the downside is that if/when they sell the house, you're looking even more like a grow-op. It's not that big a deal, but it's unusual and thus may make the house hard to sell unless you find someone else who wants a DC in their basement.
Also note that this is like $10 of electricity per day. They might want to look into getting a co-located server in a facility or something. Residential electricity is expensive.
posted by GuyZero at 11:04 AM on July 11, 2009
This is actually really good advice, but the downside is that if/when they sell the house, you're looking even more like a grow-op. It's not that big a deal, but it's unusual and thus may make the house hard to sell unless you find someone else who wants a DC in their basement.
Also note that this is like $10 of electricity per day. They might want to look into getting a co-located server in a facility or something. Residential electricity is expensive.
posted by GuyZero at 11:04 AM on July 11, 2009
Best answer: Really the advice upstream has been pretty solid, if you need numbers crunching the most efficient way to get it done is to approach it from a cloud computing perspective. EC2 and other cloud computing apps are basically in a race to sell you processor cycles as cheaply as it can be done, which is good for you. If these are mostly power users, let's say a graphics art or small video effects shop and you have half a dozen high end workstations and half a dozen machines rendering you're probably going to want to go with the big circuits+battery backup deal.
So expanding on the above, I'd recommend getting 200A service to the house and 100A brought down to a subpanel for the room where the computers live, make that subpanel probably a 6/12 or 12/24 and run your big circuits to that along with any other "normal" usage for the room. On that sub-panel install a whole panel surge protector and make sure you have someone who is experience do the work.
Finally make up a list of requirements and growth expectations over the next 2 years or so and discuss them with no less than 3 electricians and ask for multiple bids. Electricians may not help with the heating/cooling issues, but they will know how to deliver you the power in a way that is safe, flexible and that can scale. They will also know what limits exist in the building codes in the area.
GuyZero is right, this is going to be expensive to power, and even more than that cool, every watt of power you feed in to a space has to be cooled and there's no worse situation than ending up with all of your critical machines in a home and having the A/C go out on a hot summer day (and then having to halt work) or have mandatory rolling brown or blackouts as we've had in california for the last several years.
posted by iamabot at 11:29 AM on July 11, 2009
So expanding on the above, I'd recommend getting 200A service to the house and 100A brought down to a subpanel for the room where the computers live, make that subpanel probably a 6/12 or 12/24 and run your big circuits to that along with any other "normal" usage for the room. On that sub-panel install a whole panel surge protector and make sure you have someone who is experience do the work.
Finally make up a list of requirements and growth expectations over the next 2 years or so and discuss them with no less than 3 electricians and ask for multiple bids. Electricians may not help with the heating/cooling issues, but they will know how to deliver you the power in a way that is safe, flexible and that can scale. They will also know what limits exist in the building codes in the area.
GuyZero is right, this is going to be expensive to power, and even more than that cool, every watt of power you feed in to a space has to be cooled and there's no worse situation than ending up with all of your critical machines in a home and having the A/C go out on a hot summer day (and then having to halt work) or have mandatory rolling brown or blackouts as we've had in california for the last several years.
posted by iamabot at 11:29 AM on July 11, 2009
Best answer: First they should buy a few basic electrical testing tools from Home Depot and make a map of their house's circuits. Draw a quick and dirty floor plan of the house and mark all the locations of double-receptacle 120V wall sockets on it. Have one person flip breakers in the panel while the other person plugs the $5 glowing tester into the sockets to map the circuits.
If they have two 15A circuits in the basement it should not be too expensive or difficult to approximately balance half of the PCs on each one.
They should buy a kill-a-watt and determine the absolute maximum power draw of each PC separate from its display. This means running something like prime95 or dnetc on the cpu, while running a memory benchmark, and a disk benchmark like iometer on all of the drives simultaneously. To determine absolute peak the system should also burn a DVD since a DVD burner can use about 10W while in use.
Measure the displays by kill-a-watt separately from the PCs. Total them together and do not exceed 85-88% of the total capacity of a circuit. As a very rough rule you can run a 1800W hair dryer on a 120V 15A circuit assuming that nothing else is plugged in, maybe 1 light bulb, before tripping the breaker. This does not mean you should run 1800W of computers+displays+switches+routers+whatever. Go for 1600W.
They should check to see if their PC/server power supplies are PFC (power factor correcting) and labeled as 80%+ plus efficiency. A lot of cheap PC power supplies such that those that come with $65 Case+PS combos are inefficient and lose a lot of power to heat. In the long term it can be worth it to put a good quality $140 power supply in each PC. Read up on Tomshardware, HardOCP and Anandtech for reviews of recent models.
As mentioned above at a certain point heat rejection will be a problem, except in winter. It helps that they're in a northern state and not Florida, GA, SC, etc. They should check prices on 24000 BTU/h split type Samsung air conditioner units and expect to buy one that is larger than their expected head output, since you can't run an AC designed for residential use on a 24/7 duty cycle. Buying a larger than needed AC will let the AC sit idle occasionally, prolonging its life, which is probably cheaper than buying a commercial rated unit that can run constantly.
posted by thewalrus at 11:37 AM on July 11, 2009
If they have two 15A circuits in the basement it should not be too expensive or difficult to approximately balance half of the PCs on each one.
They should buy a kill-a-watt and determine the absolute maximum power draw of each PC separate from its display. This means running something like prime95 or dnetc on the cpu, while running a memory benchmark, and a disk benchmark like iometer on all of the drives simultaneously. To determine absolute peak the system should also burn a DVD since a DVD burner can use about 10W while in use.
Measure the displays by kill-a-watt separately from the PCs. Total them together and do not exceed 85-88% of the total capacity of a circuit. As a very rough rule you can run a 1800W hair dryer on a 120V 15A circuit assuming that nothing else is plugged in, maybe 1 light bulb, before tripping the breaker. This does not mean you should run 1800W of computers+displays+switches+routers+whatever. Go for 1600W.
They should check to see if their PC/server power supplies are PFC (power factor correcting) and labeled as 80%+ plus efficiency. A lot of cheap PC power supplies such that those that come with $65 Case+PS combos are inefficient and lose a lot of power to heat. In the long term it can be worth it to put a good quality $140 power supply in each PC. Read up on Tomshardware, HardOCP and Anandtech for reviews of recent models.
As mentioned above at a certain point heat rejection will be a problem, except in winter. It helps that they're in a northern state and not Florida, GA, SC, etc. They should check prices on 24000 BTU/h split type Samsung air conditioner units and expect to buy one that is larger than their expected head output, since you can't run an AC designed for residential use on a 24/7 duty cycle. Buying a larger than needed AC will let the AC sit idle occasionally, prolonging its life, which is probably cheaper than buying a commercial rated unit that can run constantly.
posted by thewalrus at 11:37 AM on July 11, 2009
Response by poster: Thanks, everyone - uniformly useful advice - I'll pass it along. I'm marking all answers as best - hope it doesn't cheapen your particular response!
I don't have good info on what they've got in each box or what the power draw per box is. They have not mentioned any problems with dissipating the heat.
Hmm, I appear to have gotten a deal on hyphens today.
posted by skyscraper at 2:36 PM on July 11, 2009
I don't have good info on what they've got in each box or what the power draw per box is. They have not mentioned any problems with dissipating the heat.
Hmm, I appear to have gotten a deal on hyphens today.
posted by skyscraper at 2:36 PM on July 11, 2009
Would it be wrong to just wait until a breaker trips to do something about it?
posted by smackfu at 3:33 PM on July 11, 2009
posted by smackfu at 3:33 PM on July 11, 2009
« Older How can I find out who wrongly gardened my garden? | What is the Name of This Tornado Doc? Newer »
This thread is closed to new comments.
Just a server, or a node in their pseudo-cluster.
This doesn't exactly answer the question, but is using cloud computing resources an option for them? Amazon (and others) have on-demand computing with as many nodes as you want. I have to imagine that their electric bill is pushing them close to the break-even point, where it's cheaper to pay for time on a cluster.
posted by chrisamiller at 10:39 AM on July 11, 2009