How do I limit/moniter net usage?
August 17, 2008 7:51 PM Subscribe
How do I monitor and/or limit Internet usage?
I'm allowed 20GB up+down per month with my ISP. I get charged extra fees for everything over and above that. My ISP gives me access to my total online, but that doesn't help me find out who has used how much. It also lags behind by a day, so if someone uses a couple GB in a day, I only find out after the fact. How can I track and/or limit how much each device is using?
Currently, I've got a Linksys WRT600N router that I would like to keep on stock firmware (it is serving files via the usb port, which DD-WRT cannot completely do yet), a D-Link 524 802.11g router, and a low powered dual core Intel Ubuntu box running MythTV which might be able to take over routing duties (if it doesn't slow down the media watching, I don't encode anything there).
I'd rather not have another computer running 24/7, so IPCop is out.
I'm allowed 20GB up+down per month with my ISP. I get charged extra fees for everything over and above that. My ISP gives me access to my total online, but that doesn't help me find out who has used how much. It also lags behind by a day, so if someone uses a couple GB in a day, I only find out after the fact. How can I track and/or limit how much each device is using?
Currently, I've got a Linksys WRT600N router that I would like to keep on stock firmware (it is serving files via the usb port, which DD-WRT cannot completely do yet), a D-Link 524 802.11g router, and a low powered dual core Intel Ubuntu box running MythTV which might be able to take over routing duties (if it doesn't slow down the media watching, I don't encode anything there).
I'd rather not have another computer running 24/7, so IPCop is out.
I haven't used it myself (yet), but bandwidthd sounds like it's exactly what you need. Might be worth checking out.
Unless someone is aware of a way to get the Linksys of D-link wireless routers to do fancier stuff than I'm aware of, your only real option is going to be to somehow involve the MythTV box. Whatever monitoring you use is going to need to see every bit that gets routed for the Internet, but ignore internal LAN traffic. So it basically needs to be 'seeing' whatever your router is seeing. In theory, you could set the Mythbox up to listen in promiscuous mode to the traffic coming into your router via a hub, but it sounds like it might be easier to just set the MythTV box up as a router. (It would need dual NICs, though. At least, for wire-based routing.) Routing uses very little in the way of system resources.
By the way, do you use static IPs? Having IPs change often would make things much more confusing. You could probably track by MAC, but I don't know if bandwidth will do that.
posted by fogster at 8:12 PM on August 17, 2008
Unless someone is aware of a way to get the Linksys of D-link wireless routers to do fancier stuff than I'm aware of, your only real option is going to be to somehow involve the MythTV box. Whatever monitoring you use is going to need to see every bit that gets routed for the Internet, but ignore internal LAN traffic. So it basically needs to be 'seeing' whatever your router is seeing. In theory, you could set the Mythbox up to listen in promiscuous mode to the traffic coming into your router via a hub, but it sounds like it might be easier to just set the MythTV box up as a router. (It would need dual NICs, though. At least, for wire-based routing.) Routing uses very little in the way of system resources.
By the way, do you use static IPs? Having IPs change often would make things much more confusing. You could probably track by MAC, but I don't know if bandwidth will do that.
posted by fogster at 8:12 PM on August 17, 2008
Tomato Firmware has a pretty slick bandwidth monitor. Dunno about its USB capabilities though.
posted by meta_eli at 8:16 PM on August 17, 2008
posted by meta_eli at 8:16 PM on August 17, 2008
Oh! I missed you are already running Ubuntu! That makes it all easier ... I would do this:
Get a router that doesnt do wifi.
Plug the router into a 10/100 *hub*
Plug your 802.11g router into the hub, make sure it is BRIDGING, not routing.
Plug your WRT into the router (for higher speed?)
Plug your PC into the hub (or get a second nic for monitoring).
You can now passively sniff traffic on the HUB, and trace who exactly is using what. There are quite a few Linux tools that can monitor things, but iptables + MRTG would be the easiest.
The reason you need the PC going into the *hub* (not switch) is so you can see ALL of the network traffic hitting the router/internet. The PC can see all the internal IP addresses, and monitor exactly what is going in and outbound by IP. Anything you dont care about (ie, the TV loading off the WRT) can be on a switch, as you don't need to log that traffic.
NTOP might be overkill, but it works well.
posted by SirStan at 8:18 PM on August 17, 2008
Get a router that doesnt do wifi.
Plug the router into a 10/100 *hub*
Plug your 802.11g router into the hub, make sure it is BRIDGING, not routing.
Plug your WRT into the router (for higher speed?)
Plug your PC into the hub (or get a second nic for monitoring).
You can now passively sniff traffic on the HUB, and trace who exactly is using what. There are quite a few Linux tools that can monitor things, but iptables + MRTG would be the easiest.
The reason you need the PC going into the *hub* (not switch) is so you can see ALL of the network traffic hitting the router/internet. The PC can see all the internal IP addresses, and monitor exactly what is going in and outbound by IP. Anything you dont care about (ie, the TV loading off the WRT) can be on a switch, as you don't need to log that traffic.
NTOP might be overkill, but it works well.
posted by SirStan at 8:18 PM on August 17, 2008
Most routers give out pretty static DHCP leases these days -- I think ive had the same IP from my WRT54G for the past 2 years.
A solution like AnalogX will show combined traffic unfortunately -- somethign the OP most likely doesnt want (ie, local traffic copying off the WRT, as well as internet traffic).
You need something between the router and the users for this to monitor appropriately.
posted by SirStan at 8:20 PM on August 17, 2008
A solution like AnalogX will show combined traffic unfortunately -- somethign the OP most likely doesnt want (ie, local traffic copying off the WRT, as well as internet traffic).
You need something between the router and the users for this to monitor appropriately.
posted by SirStan at 8:20 PM on August 17, 2008
It looks like bandwidthd might not work if it's running on the router, actually. It works via libpcap, a packet-sniffing library. So SirStan's suggestion for the hub might work better.
In all seriousness, though, can you still buy hubs? I'm having a hard time finding them. Trying to overload a switch's CAM table via ARP spoofing might work, but I wouldn't want to rely on it long-term.
posted by fogster at 8:27 PM on August 17, 2008
In all seriousness, though, can you still buy hubs? I'm having a hard time finding them. Trying to overload a switch's CAM table via ARP spoofing might work, but I wouldn't want to rely on it long-term.
posted by fogster at 8:27 PM on August 17, 2008
Response by poster: I do have a hub kicking around. Should have listed it among my assets. I think it's only 10baseT though. I don't think I'll need it, anyway.
So it looks like I'll either move the USB drive to the MythTV box and flash the router or stick a NIC in the MythTV box and install some monitoring (NTOP?) and routing (suggestions?) packages.
I'm leaning towards the second option. I'm glad the routing won't take up much in the way of resources. The MythTV box is solid state already, so it shouldn't be a problem to leave it running. I should be able to differentiate between the internal vs. external traffic without the hub, right?
On the other hand, I might have to move the USB file storage anyway (can't stream reliably off of it yet, don't know why).
Further thoughts?
posted by ODiV at 8:44 PM on August 17, 2008
So it looks like I'll either move the USB drive to the MythTV box and flash the router or stick a NIC in the MythTV box and install some monitoring (NTOP?) and routing (suggestions?) packages.
I'm leaning towards the second option. I'm glad the routing won't take up much in the way of resources. The MythTV box is solid state already, so it shouldn't be a problem to leave it running. I should be able to differentiate between the internal vs. external traffic without the hub, right?
On the other hand, I might have to move the USB file storage anyway (can't stream reliably off of it yet, don't know why).
Further thoughts?
posted by ODiV at 8:44 PM on August 17, 2008
The hub recommendation was if you didn't want to use MythTV as your router. The system running the stats needs to 'see' all the traffic that the router sees, and having it share a hub and enter promiscuous mode.
I'm going to leave routing to the experts. I've done it on OpenBSD, but it's different under Linux. And probably, much easier... I think iptables is the norm on Linux. (As an aside, you can use setting up a 'real' computer as a router as an opportunity to do cool stuff like set up traffic shaping or a proxy server, if desired.)
Is the "USB file storage" just a USB hard drive being shared over the LAN? You could probably use something like Samba on the Linux box to share it that way, if I understand you correctly.
posted by fogster at 9:19 PM on August 17, 2008
I'm going to leave routing to the experts. I've done it on OpenBSD, but it's different under Linux. And probably, much easier... I think iptables is the norm on Linux. (As an aside, you can use setting up a 'real' computer as a router as an opportunity to do cool stuff like set up traffic shaping or a proxy server, if desired.)
Is the "USB file storage" just a USB hard drive being shared over the LAN? You could probably use something like Samba on the Linux box to share it that way, if I understand you correctly.
posted by fogster at 9:19 PM on August 17, 2008
If you are at all into scripting....
If you flash with DD-WRT, you can setup a series of iptables rules taht will keep a "byte count" per rule.
So you make a bunch of "allow" rules for each IP. You can then query that list, and MRTG the results, or whatever suits your fancy.
Something needs to "see" all of the traffic. If your router does it -- great. If not, you need a hub for the Linux Compy to see all of the traffic. You might decide to make the Linux compy the router as well (which would make the mrtg game MUCH easier).
posted by SirStan at 9:29 PM on August 17, 2008
If you flash with DD-WRT, you can setup a series of iptables rules taht will keep a "byte count" per rule.
So you make a bunch of "allow" rules for each IP. You can then query that list, and MRTG the results, or whatever suits your fancy.
Something needs to "see" all of the traffic. If your router does it -- great. If not, you need a hub for the Linux Compy to see all of the traffic. You might decide to make the Linux compy the router as well (which would make the mrtg game MUCH easier).
posted by SirStan at 9:29 PM on August 17, 2008
Netlimiter has several versions which might fill your needs. I use the free version which tracks up/download numbers by hour, day, month, year.
Netlimiter.com
posted by bbranden1 at 10:30 AM on August 18, 2008
Netlimiter.com
posted by bbranden1 at 10:30 AM on August 18, 2008
Response by poster: Thanks for the help!
I installed dhcp3 server, firestarter (for GUI goodness), and ntop on the Ubuntu box.
ntop's web interface is looking pretty cool. Firestarter made setting up the routing easy. I might need to look into some permanent leases based on MAC though (I'm not sure how much everyone's address will change.
I'm running ntop on startup with -b -n -z -d -L and -i "eth0" which I think is what I should be doing (eth0 is the gigabit that will go to the Linksys router, eth1 is the new NIC which is connected to the Internet). I'm not entirely clear what the -L does, but I'll find out soon enough.
Now I just need to find out how to ignore local traffic. I'll come back and post what I've come up with after that.
posted by ODiV at 3:11 PM on August 18, 2008
I installed dhcp3 server, firestarter (for GUI goodness), and ntop on the Ubuntu box.
ntop's web interface is looking pretty cool. Firestarter made setting up the routing easy. I might need to look into some permanent leases based on MAC though (I'm not sure how much everyone's address will change.
I'm running ntop on startup with -b -n -z -d -L and -i "eth0" which I think is what I should be doing (eth0 is the gigabit that will go to the Linksys router, eth1 is the new NIC which is connected to the Internet). I'm not entirely clear what the -L does, but I'll find out soon enough.
Now I just need to find out how to ignore local traffic. I'll come back and post what I've come up with after that.
posted by ODiV at 3:11 PM on August 18, 2008
Response by poster: I've got the following line in my rc.local:
ntop -b -c -g -n -z -d -L -i "eth0"
-c is listed as "Use this parameter to prevent idle hosts from being purged from memory." and
-g is "Use this parameter to tell ntop to capture data only about local hosts."
but neither seem to be working. If someone doesn't access the network for a certain amount of time, then their host doesn't show up on the webage. Also, it's still tracking non-local hosts. I restarted after adding the line to rc.local. Is that not a good way to start ntop up again with the new parameters?
Took a look at MRTG as well, but that doesn't seem to be designed for what I need. I might need to do that iptable scripting after all.
And then after I get either ntop or the iptables doing what I want, I have to figure out the whole Samba thing (and also how to ignore the traffic to/from that, erg!)
posted by ODiV at 8:46 PM on September 8, 2008
ntop -b -c -g -n -z -d -L -i "eth0"
-c is listed as "Use this parameter to prevent idle hosts from being purged from memory." and
-g is "Use this parameter to tell ntop to capture data only about local hosts."
but neither seem to be working. If someone doesn't access the network for a certain amount of time, then their host doesn't show up on the webage. Also, it's still tracking non-local hosts. I restarted after adding the line to rc.local. Is that not a good way to start ntop up again with the new parameters?
Took a look at MRTG as well, but that doesn't seem to be designed for what I need. I might need to do that iptable scripting after all.
And then after I get either ntop or the iptables doing what I want, I have to figure out the whole Samba thing (and also how to ignore the traffic to/from that, erg!)
posted by ODiV at 8:46 PM on September 8, 2008
Response by poster: Okay, so I definitely had not stopped it properly or something. Just killed the daemon and restarted it with the above command. Seems to be working fine: tracking local hosts only and keeping them indefinitely.
Next up: Samba and how to avoid tracking the data that flows from the samba share to the local hosts. Any tips (if anyone's reading) would be appreciated.
posted by ODiV at 10:22 PM on September 8, 2008
Next up: Samba and how to avoid tracking the data that flows from the samba share to the local hosts. Any tips (if anyone's reading) would be appreciated.
posted by ODiV at 10:22 PM on September 8, 2008
« Older How to help a parent and her adult children cope... | Should I stay or should I go now? Newer »
This thread is closed to new comments.
http://hetos.de/bwlog.html
Or -- build a low power PC (You can build one that is solid state, and probably uses less power than the WRT600N) and use Smoothwall and iptables (or something more elaborate if it has it) for ~$150?.
Or try smoothwall in VMWare on the desktop with a couple more nic cards in it? It shouldnt take too much CPU time.
posted by SirStan at 8:10 PM on August 17, 2008