Weekly News 2002-11-10

Mnet Hack Day

The Mnet Hack Day on 2002-11-10 was a success, despite the traditional attack of bad luck including no hot water, no e-mail service, an arm still being broken, and a viral infection. Various people showed up and much of the progress reported below was accomplished on that day.

progress

Arno Waschk implemented a percent-complete progress meter for the new download system. This eliminated one of the remaining open bugs that are blocking the v0.6-STABLE release. Details in (select "v0.6" in the "Group" listbox and click the "Browse" button to see bugs that are holding up the v0.6-STABLE release.)

The erasure code that Mnet uses has a bug which occasionally causes corrupted blocks. These corrupted blocks are immediately detected since all blocks are named by their SHA1 hash. This week we added a workaround -- if a corrupted block is detected, then redownload the block using a different combination of shares. The next version of the Mnet distributed filestore format will drop the buggy erasure code entirely and switch to a widely used erasure code and implementation by Luigi Rizzo. Details in .

Mnet has two GUIs, an HTML-based GUI served up by the Mnet node acting as a local web server, and a wxWindows-based GUI which has all the modern conveniences of buttons, live updating of data and so forth. The former GUI is little-loved by the developers, but we're making sure that it continues to work until someone writes a nice new one based on the v0.7 code base. The HTML-based GUI has been broken in CVS for the last couple of weeks (since the new download system was added and the old one removed), and this week we fixed it so you can download again. Details in .

We made some progress this week on the "v0.7" revamp of the codebase. More of the code has been reformatted to fit the new coding standard, and more unit tests pass now.

Darius Bacon built Mnet from source, and it didn't run the first time because it had been built with Python v2.1 and executed with Python 2.2. I tweaked the makefile so that it rebuilds any extension modules which were not built with your chosen version of Python, and it changes the start script to launch your chosen version of Python. I also renamed the start script from "Broker" to "Mnet.sh", although the docs have not been updated to reflect the change. Details in .

issues

Mnet has long been plagued by memory leaks. Recently we added memory leak detection tests to some of the unit tests, and on different developer's machines these tests sometimes claim to have found a memory leak. This might be an artifact of the statistical technique used. Before we look for a memory leak, we need to turn up the "numsamples" and "iterspersample" parameters to reduce statistical variability. (Those parameters are turned down to minimal in the distribution in order to make unit tests finish faster.) Details in .

hacks needed

Mnet has a download system with two components. One component, named BlockWrangler, manages all of the logic which is required to have a correct download. The other component, named BlockWranglingStrategy, gets to make "strategic", or heuristic, decisions about which kind of peers you prefer to download from first, and how many simultaneous downloads you want to initiate. The current BlockWranglingStrategy is named "GSR" for Greedy, Simple, Random. It was intended mostly as a placeholder while developing the BlockWrangler, but it works well enough that people continue to use it. However, it has several serious shortcomings. I encourage people to write their own BlockWranglingStrategies and submit them for inclusion in the distribution. Details in .