June 22, 2010

Summer of Code Progress: Merging Launchpad branches

Filed under: Blog — krkhan @ 5:21 pm

Related Links

Summer of Code Archive Inspirated Code
Report Guidelines Ubuntu Wiki
Original Proposal Ubuntu Wiki

Time Spent

40 hours.


This week was spent on trying to get my Launchpad branches merged upstream. During the process many concerns were raised which resulted in a number of patches and discussions.


Quoting Stuart Bishop’s response from Launchpad-dev:

I’m really not sure of the best way to tackle this problem. The
Librarian data is not stored in the database because there are
multiple TB of files. The team membership information is in the
relational database. There are no indexes anywhere to the contents of
the Librarian files. I think we need some sort of external search
engine (I don’t think we don’t want to integrate this into the
Librarian core). Ideally we could feed it subscriber information
allowing it to determine the set of 32000 attachments that ubuntu-bugs
has access to rather than having to calculate this information from
the relational db and then feed the ids to the search engine.

Whatever approach certainly needs signoff from the LP team leads, as
the resource requirements are non trivial and someone needs to pay for
the hardware.

Waiting Items


Stalled Items

  • Implementation of Bug.findAttachments().


  • Merge Proposal: Got the export-Person-getBugSubscriberPackages branch approved after fixing various tests and bugs.
  • Merge Proposal: Implemented Horspool’s algorithm and fixed various bugs in the implement-Bug-findAttachments branch. The branch itself didn’t get approved because of its design approach for searching the attachments:

    I’m going to mark this review as ‘disapproved,’ not because the code is
    bad (it isn’t) but because I don’t think this is the right solution to
    the problem. I’m sorry to say that I don’t know what the right solution
    to the problem actually is at this point, but I’d guess that something
    involving FTIs would be a start, or some kind of asynchronous processing
    of searches (though then you get into all kinds of knotty stuff with

Minor Tasks

For reading the file in chunks, I took the Wiki code for Horspool algorithm, converted it to Python and modified a little so that it would work with stream files.

Actions for the Following Report

There doesn’t appear to be a straightforward efficient way for searching bug attachments. I’ll discuss the course of my future development with Bryce tonight and decide whether I should head over to Arsenal development or should I focus on the proposed (albeit germinal) solutions from IRC and the mailing list.

Tags: , , , , , , , ,

No Comments

No comments yet.

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.