[mnet-devel] another download strategy

[Zooko]

Here's an alternative.  We still have the "locate and download blocks rate" 
measure as described in the previous attempt [1].

Keep a sorted list of blockservers, sorted by "locate and download blocks rate".

Whenever you want to download a block, you do so from the highest-ranked server 
from that list that has a located block.

Whenever you want to search for blocks, you do so from the highest-ranked server 
from that list that you haven't already searched for those blocks.

Now this has the funny property that when you choose a server to search for 
blocks, you ignore the XOR metric!  You don't go to a server that is close to 
any particular the block in the XOR space, but instead you go to your favorite 
server.

This is reminiscent of the joke about the drunk who lost his key, and looks for 
it under the streetlight, not because that's where he lost it, but because 
that's where it would be easiest to see.

If there are a lot of keys (blocks), and if they are evenly distributed, and if 
you need only a fraction of them, then this looks like a good strategy!

...

But if there is only one block (e.g. an inode block), then this is a poor 
strategy.

This suggests that the blockwrangler should remember which blocks are part of 
which file.  There are three ways blockwrangler can use this data:

1.  If the block is 1-of-1 (e.g. an inode or a small file that fits into a 
single data block), then blockwrangler should treat it differently.

2.  If you are downloading two files, each of which is 1000-out-of-4000 blocks, 
and he has already sent a "request block" request for 1100 different blocks of 
the first file, and he's already gotten 900 successful responses, then he should 
really not be sending out more "request block" requests for the first file, but 
instead should send some out for the second file.

3.  "Primary" shares are better than "secondary" in two ways:

3.a.  Primary shares can be viewed incrementally, before the file is complete.
3.b.  Primary shares don't cost a bunch of CPU churning when used to reconstruct 
the complete file.

So overall I like this strategy a lot better than the previous one.  It's 
simpler and I *feel* like it'll perform better, but it might perform badly on 
inode blocks and single-block-files without some added wrinkle.

Regards,

Zooko

[1] http://sourceforge.net/mailarchive/forum.php?thread_id=1892147&forum_id=7702

http://zooko.com/
         ^-- under re-construction: some new stuff, some broken links



-------------------------------------------------------
This SF.net email is sponsored by:
The Definitive IT and Networking Event. Be There!
NetWorld+Interop Las Vegas 2003 -- Register today!
http://ads.sourceforge.net/cgi-bin/redirect.pl?keyn0001en
_______________________________________________
mnet-devel mailing list
mnet-devel at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mnet-devel