02/02/2005

Cluckfuck McDuck, Bumfuck Chuckschmuck

This is the fifth in a series of back-dated site updates that have only just been published

firewalling and traffic shaping part 2

So we move onto traffic shaping. The main reason I want to do this is so that I can still log in to remote hosts using ssh and be able to type fairly easily, even if I am downloading something or you fucks are spamming my web server with stupid requests (e.g. "GET /favicon.ico". Note to self: write another rant about Firefox another time)

Suppose you're uploading and also trying to ssh. If you're downloading you're also uploading; there are things called ACK packets to maintain a reliable stream. ssh sets its outgoing packets to have Type Of Service (TOS) Minimise-Delay so they leave your router at high priority by default. However they also are transmitted at the speed your network card talks to your modem, typically 10 megabits per second. This is way above your internet connection's upload speed, which is typically about 256 kilobits per second, if you're on ADSL. So a queue forms in the modem. You don't want this. The modem will just drop packets it doesn't have room to queue, for example ssh packets that should be prioritised over, say, http ACK packets. This creates latency and those annoying delays between keypress and character appearing in your ssh window. You want to

When there is network traffic to send out the kernel puts it in a Queueing Discipline, abbreviated to qdisc. Then the packets are dequeued as determined by the qdisc. A classless qdisc treats all packets the same. A classful qdisc is split into classes and you have to specify filters to tell the qdisc which packets go into which class. The default qdisc on an ethernet interface is a first in first out 3 band priority queue, that is, when a packet is queued it gets put into one of three FIFO queues depending on its TOS value. Packets are dequeued by priority and no attempt is made to slow the connection down; if there's packets to send, they'll just go.

Probably the easiest way to do what I want is to replace the default qdisc with a Hierarchial Token Bucket (HTB). This qdisc is pretty much designed for the problem I'm trying to solve but is only available in recent kernels (2.4.20 onwards, so not in Debian Woody unless you've upgraded) What you can do with this is set the maximum rate at which the entire qdisc can dequeue (that is at which your upstream link can transmit), and classify traffic into subflows which will be dequeued in an order given by choosable priorities. The best part is that if a higher priority queue isn't full lower priority queues can borrow their bandwidth.

(If you don't have a recent kernel you can make a similar setup by putting a bunch of Token Buckets into a simple Priority queue; the token buckets are independent of each other and their rates have to add up to the total upstream bandwidth you have)

Implementation

Okay that's the theory, and it's not really conceptually difficult, my terrible explanations notwithstanding. How do you actually set up this stuff? Obviously you need it compiled into your kernel. To initialise it however requires use of the command tc, which has probably one of the most awful command line interfaces ever made. I don't even think it's meant to be human-readable.

The first part is relatively straightforward, you set the qdisc of your outgoing interface to be an HTB, and make its first child class specify the maximum bandwidth of the link. (That should be a little bit less than your actual upstream bandwidth) Then you create subclasses of that first class that have a given guaranteed bandwidth, a given maximum bandwidth, and a priority. Obviously the given maximum bandwidth for each subclass shouldn't exceed the maximum bandwidth of the the parent class, nor should the sum of the guaranteed bandwidths.

Finally you attach filters to the qdisc that say which subclass of the qdisc to put packets into. This is probably the hardest part as the u32 filter is an awful thing that requires intimate knowledge of the structure of the header of a packet and inhuman ability with bitshifts and bitmasks and stuff. I found it much easier to use the fwmark filter, which classifies packets according to their iptables mark. This is why I have the mangle table, as I mentioned in the previous update. It says a lot about tc that writing iptables rules is much easier(!)

Summary

Anyway so basically what you have happening is this

Please note you can only do this with outgoing packets. There's not a lot you can do with incoming traffic, except police it. That is, drop some if you recieve too much. TCP ensures it gets retransmitted at the other end. I do this but I'm not sure how much effect it has.

Links