Jeremy Gore wrote:

> I don't know whether there have been any such significant changes to
> the source (I kind of doubt it). If you could switch to an earlier
> release (ask bram for one I guess) and test it out that would be the
> place to start.

People have been commenting about relative performance between 3.1 and
3.0.2 a lot. This is a very complicated issue, so I'll explain in
detail. At the bottom I'll explain what I'm going to do about the new
problems, they're all quite fixable.

There are four things which 3.1 has over 3.0.2 which are likely to alter
performance.

(1) Better rate estimation

The old method of calculating transfer rates had some nasty artifacts and
tended to inflate transfer rates. The new algorithm is much better, and
also the default slice size was reduced from 32k to 16k, which doubled the
effective sampling rate. These changes will produce subtly better overall
performance, but somewhat lower (although much more accurate!) estimated
transfer rates.

(2) rarest first

Previously, clients downloaded pieces in random order. This caused several
problems. For one thing, in the case of a deployment which was
bottlenecking on the upload rate of the origin, the download would go much
slower than necessary because the origin would frequently send out the
same piece several times. For another, the likelihood of some piece
getting lost was rather high, so the origin going down frequently made a
file unavailable.

Rarest first downloads pieces starting at the least common. These are the
pieces which both are most likely to be unavailable later and are most
likely to be useful for uploading to peers.

After 3.1 was released, there was a sudden and dramatic improvement in the
robustness and longevity of BitTorrent deployments. I think this is mostly
due to rarest first (and (4) below, which rarest first enabled).

Unfortunately rarest first exacerbates a problem which was happening
before. When a downloader first connects, especially at the very start of
a very popular file, they have nothing to upload, so they sit there choked
by everyone until they manage to cobble together a whole piece from random
optimistic unchokes and then can start uploading, at which tit-for-tat
works its magic and they start getting a decent transfer rate.

Rarest first is arguably the worst possible strategy at the very
beginning, because it maximizes the amount of time it will take to
download that one piece. This is somewhat mitigated by (4) below, which
increases the chances that a whole piece will be transferred during a
single optimistic unchoke period, but it's still a serious problem,
especially at the beginning of a very popular download.

Of course, random sometimes happens to pick the rarest first, so the old
behavior wasn't especially good at the beginning either.

(3) anti-snubbing

Watching transfers happen with diagnostics turned on, I noticed that
occasionally all peers in 69 happen to choke at the same time, resulting
in a very slow download rate until the optimistic unchoke finds better
ones. To mitigate this, I added anti-snubbing, which checks to see if one
of the peers it would like to reciprocate with hasn't sent anything in
over a minute, and in that case dedicate that slot to an optimistic
unchoke.

This technique doesn't kick in unless poor download rates are being
experienced, and in those cases it does at least twice as much optimistic
unchoking as it does normally. The result is much more consistent download
rates for everyone.

Due to the evening out effect this produces, some people may experience
reduced download rates because the idiosynchracies of their connection
happened to make them get the better of the somewhat more arbitrary load
distribution which was happening before. It's hard to say when this might
happen though.

A much more serious problem this produces has to do with how I implemented
it. Previously, once a downloader finished they preferred peers which they
uploaded to the fastest. Now once a peer is finished it simply figures
it's snubbed by everyone, and unchokes four different peers every
period. This results in greatly reduced upload rate because connections
have to slow start once every 30 seconds, and also because peers on very
fast connections may happen to unchoke only peers on much slower
connections in any given period, bottlenecking on the download
side. Setting max_uploads to 20 helps with this problem but doesn't
completely make it go away.

I think this is the largest performance regression to happen in the latest
release, since it drastically reduces upload capacity utilization.

(4) reduced default piece size

The default piece size was reduced in the latest release from a megabyte
to a quarter megabyte. This won't make any difference for .torrents which
were generated with the old release, but for ones generated with the new
one it will result in a much shorter slow period at the very beginning and
subtly better performance at other times.

This increases on the wire overhead somewhat, but it's still well below
one percent. The main danger is that the risk of a piece getting lost is
much greater if all the clients are 3.0.2, but with rarest first widely
implemented that isn't a concern.


Those are the new problems, here are the planned solutions -

(1) Rarest first will be kept, but when no pieces have been downloaded yet
instead of picking pieces which only one peer has, it will try to pick
pieces which about half of all of its peers have, these should both be
reasonably quick to download and useful for uploading, since a fair number
of peers have them and a fair number of peers want them.

(2) Upload behavior after completion will be changed to favor greater
upload rates again.

(3) A new feature will be added to complete uploaders that they give each
new downloader which connects at least a full piece before choking them.
This will help speed up the initial slow phase a lot. This is obviously
gameable, but it's always been the case that you can get more out of a
complete downloader by making lots of connections to it, and the way to
deal with that is to make complete downloaders view peers with the same IP
address as effectively the same connection for unchoking purposes.

That should clean up all the new artifacts nicely. The next release will
in all ways be clearly better than both 3.0.2 and 3.1.

-Bram