Merging two internet connections niftily?
October 10, 2010 8:35 PM Subscribe
One linux box, 3 NICs, two internet connections, one local network. Help me make it be the best it can be.
I recently got sick of maxing out around 350 K/s with DSL and decided to order 15 mbit cable (I've tried higher speed DSL, but the physical infrastructure to my apartment choked and they had to turn it back down and refund me).
Here's my catch - I don't want to lose the DSL. I own a domain name pointed at that IP address and my mail server is in my apt. My DSL provider is all sorts of nice and friendly, and leaves my port 25 (and all of the others well enough alone). I want to keep the DSL and simply add the cable connection to my internet world.
I already have a linux box with three NICs, and one of them is unused. It is already my gateway and dhcp server (as well as being my mail server), and I already have some traffic shaping using TC set up.
Is it AT ALL possible to set things up so that clients on my local network will use the cable connection as much as possible, but can be load balanced to the DSL when things are getting saturated? Some simple googling last night turned up something that looked to round-robin outgoing connections (and therefore, I would guess, roughly split the resultant incoming connections), but since the cable should be much faster than the DSL I don't want to do that either.
I'm quite willing and able to find new bits of software and build from source, but my installation is fairly out of date (kernel 2.6.19.21). And, of course variations on my proposed plan are welcome. Thanks!
I recently got sick of maxing out around 350 K/s with DSL and decided to order 15 mbit cable (I've tried higher speed DSL, but the physical infrastructure to my apartment choked and they had to turn it back down and refund me).
Here's my catch - I don't want to lose the DSL. I own a domain name pointed at that IP address and my mail server is in my apt. My DSL provider is all sorts of nice and friendly, and leaves my port 25 (and all of the others well enough alone). I want to keep the DSL and simply add the cable connection to my internet world.
I already have a linux box with three NICs, and one of them is unused. It is already my gateway and dhcp server (as well as being my mail server), and I already have some traffic shaping using TC set up.
Is it AT ALL possible to set things up so that clients on my local network will use the cable connection as much as possible, but can be load balanced to the DSL when things are getting saturated? Some simple googling last night turned up something that looked to round-robin outgoing connections (and therefore, I would guess, roughly split the resultant incoming connections), but since the cable should be much faster than the DSL I don't want to do that either.
I'm quite willing and able to find new bits of software and build from source, but my installation is fairly out of date (kernel 2.6.19.21). And, of course variations on my proposed plan are welcome. Thanks!
Forget load balancing... it's not worth the effort. Round robin is doable.
If you use packet marking and multiple routing tables, you can get packets in from either internet source to go back out the same source (which is the tricky bit) and have everything work the way you want.
It's definitely NON-TRIVIAL and will cause you a few days worth of head scratching assuming you are familiar with IP routing, subnet masks, etc.
You'll need the ip_conntrack module, so that you can do iptables -j CONNMARK type commands, to set flags on the inbound packets as to their source.
If you do this right, any connection starting on the DSL side (either way) stays that way... and any connection starting with the cable stays that way as well.
It took a lot of work to get it done, but it's what we use at my day job.
posted by MikeWarot at 9:41 PM on October 10, 2010
If you use packet marking and multiple routing tables, you can get packets in from either internet source to go back out the same source (which is the tricky bit) and have everything work the way you want.
It's definitely NON-TRIVIAL and will cause you a few days worth of head scratching assuming you are familiar with IP routing, subnet masks, etc.
You'll need the ip_conntrack module, so that you can do iptables -j CONNMARK type commands, to set flags on the inbound packets as to their source.
If you do this right, any connection starting on the DSL side (either way) stays that way... and any connection starting with the cable stays that way as well.
It took a lot of work to get it done, but it's what we use at my day job.
posted by MikeWarot at 9:41 PM on October 10, 2010
This is how I do it.
Shorewall rocks for this; it's basically a set of scripts that use the normal iptables etc modules, but allow you to configure it in a relatively simple way.
You definitely want the TRACK option, so that packets coming in one line get the replies going back out the same line. That will solve most of your problems right there. You can also route specific outgoing traffic (say SMTP) via one particular interface using the tcrules.
You can decide to use the ADSL as some additional bandwidth, by using the balance option, and give the cable connection substantially more weight so it got most routes. It's a fairly crude route-based balancing mechanism though, so you might be better off purely using the cable route for normal traffic, and keep the adsl line purely for server services.
posted by ArkhanJG at 11:53 PM on October 10, 2010
Shorewall rocks for this; it's basically a set of scripts that use the normal iptables etc modules, but allow you to configure it in a relatively simple way.
You definitely want the TRACK option, so that packets coming in one line get the replies going back out the same line. That will solve most of your problems right there. You can also route specific outgoing traffic (say SMTP) via one particular interface using the tcrules.
You can decide to use the ADSL as some additional bandwidth, by using the balance option, and give the cable connection substantially more weight so it got most routes. It's a fairly crude route-based balancing mechanism though, so you might be better off purely using the cable route for normal traffic, and keep the adsl line purely for server services.
posted by ArkhanJG at 11:53 PM on October 10, 2010
Response by poster: Awesome, I already use shorewall as well. Will check that out!
A lot of this makes an awful lot of sense to me, but one thing I'm utterly failing to grok is how a bittorrent client will work on my local network. I'm not really sure how ingoing/outgoing works well enough in torrent-land. There's a "peer listening port" for transmission that I have a DNAT rule for, but that's for incoming connections... what generates those?
Every time I think I understand networking I start to think about it and am proven wrong.
posted by flaterik at 12:37 AM on October 11, 2010
A lot of this makes an awful lot of sense to me, but one thing I'm utterly failing to grok is how a bittorrent client will work on my local network. I'm not really sure how ingoing/outgoing works well enough in torrent-land. There's a "peer listening port" for transmission that I have a DNAT rule for, but that's for incoming connections... what generates those?
Every time I think I understand networking I start to think about it and am proven wrong.
posted by flaterik at 12:37 AM on October 11, 2010
If I were you I'd run the mail server and DNS over the DSL and everything else over the cable. The cable connection has so much more bandwidth that it really isn't worth it to use any kind of load balancing.
posted by atrazine at 1:08 AM on October 11, 2010
posted by atrazine at 1:08 AM on October 11, 2010
Best answer: Your outbound bittorrent connections will go with whatever is specified as the default route of the two (presumably, your cable connection), unless you use the balance option in providers, in which case it'll use both for outbound traffic.
As far as incoming goes;
/etc/shorewall/rules:
#ACTION SOURCE DEST PROTO DEST PORT(S) SOURCE ORIGINAL
# PORTS(S) DEST
DNAT net loc:192.168.1.3 tcp 6881
This will DNAT (port mapping) connections coming from your external, net classified connections to port 6881, to the local interface, onto the box 192.168.1.3 - obviously, substitute the correct internal IP.
That particular rule will allow traffic on either line, coming in on port 6881, to go that box. If you want a collection of ports, you can specific them with commas (i.e. 22,6881) or using hyphens for a range. You can also specify standard services with names from /etc/services, so (ssh,imap,https,6881) is functionally equivalent to (22,143,443,6881)
Note, the DNAT rule will forward the traffic; you'll still need to allow it through the firewall. I think one of the shorewall options creates an open port in the firewall to match automagically, but I always add an additional line in /etc/shorewall/rules to allow the traffic through - belt & braces!
so for example, I'd have this in /etc/shorewall/rules to send ssh and https to one box, and bittorrent incoming traffic on 6881 to another.
ACCEPT all fw tcp ssh,https,6881
DNAT net loc:192.168.3.3 tcp ssh,https
DNAT net loc:192.168.3.10 tcp 6881
Note, the accept rule applies to the shorewall firewall itself; as far as the outside world goes, it's talking to your firewall, it doesn't know about your internal network at all. That's shorewall's job!
In my /etc/shorewall/providers, because I'm loadbalancing 2 ADSL lines of equal weight
adsl1 1 1 main eth2 81.260.128.14 track,balance eth0
adsl2 2 2 main eth3 81.260.128.38 track,balance eth0
(where 128.14 and 128.38 are my upstream routers, they may be NAT ip addresses in your case, eth2 and eth3 are my adsl lines respectively, and eth0 is my internal interface)
posted by ArkhanJG at 6:32 AM on October 11, 2010
As far as incoming goes;
/etc/shorewall/rules:
#ACTION SOURCE DEST PROTO DEST PORT(S) SOURCE ORIGINAL
# PORTS(S) DEST
DNAT net loc:192.168.1.3 tcp 6881
This will DNAT (port mapping) connections coming from your external, net classified connections to port 6881, to the local interface, onto the box 192.168.1.3 - obviously, substitute the correct internal IP.
That particular rule will allow traffic on either line, coming in on port 6881, to go that box. If you want a collection of ports, you can specific them with commas (i.e. 22,6881) or using hyphens for a range. You can also specify standard services with names from /etc/services, so (ssh,imap,https,6881) is functionally equivalent to (22,143,443,6881)
Note, the DNAT rule will forward the traffic; you'll still need to allow it through the firewall. I think one of the shorewall options creates an open port in the firewall to match automagically, but I always add an additional line in /etc/shorewall/rules to allow the traffic through - belt & braces!
so for example, I'd have this in /etc/shorewall/rules to send ssh and https to one box, and bittorrent incoming traffic on 6881 to another.
ACCEPT all fw tcp ssh,https,6881
DNAT net loc:192.168.3.3 tcp ssh,https
DNAT net loc:192.168.3.10 tcp 6881
Note, the accept rule applies to the shorewall firewall itself; as far as the outside world goes, it's talking to your firewall, it doesn't know about your internal network at all. That's shorewall's job!
In my /etc/shorewall/providers, because I'm loadbalancing 2 ADSL lines of equal weight
adsl1 1 1 main eth2 81.260.128.14 track,balance eth0
adsl2 2 2 main eth3 81.260.128.38 track,balance eth0
(where 128.14 and 128.38 are my upstream routers, they may be NAT ip addresses in your case, eth2 and eth3 are my adsl lines respectively, and eth0 is my internal interface)
posted by ArkhanJG at 6:32 AM on October 11, 2010
« Older Do we live in a world where a manufacturing... | Am I gay and blocked? Bisexual? Help me understand... Newer »
This thread is closed to new comments.
You've stated your reasons that you want to keep the DSL - so make the DSL just for serving your mail and domain needs. It's been doing that up until now, so it should be simple enough to limit the interfaces your mail and domain name are using. For just generic surfing, downloads, video, etc - set your defaults to use the cable and be done with it.
posted by HannoverFist at 9:00 PM on October 10, 2010