Inspirated

“O Marvelous! what new configuration will come next? I am bewildered with multiplicity.” — William Carlos Williams

Up until now my GSoC development was dealing with the “monitoring” aspects of arm. In addition to being a monitor a Tor controller is also supposed to, well, control the Tor client. Tor offers a detailed specification which can be used to interact with a running client and control it in a myriad of ways. This specification is then used by controllers like arm and Vidalia to fetch and/or update the client configuration.

The CLI configuration panel for arm already understood the control specification quite well. The first step towards reusing those portions for the GUI was to rewrite the inheritance code so that underlying data structures of the CLI were automagically connected to the Gtk+ models. That made the actual implementation of GUI configuration panel a whole lot cleaner. As things are, the panel detects the data types of configuration options and opens pertinent dialogs for user input:

(Click on the thumbnail for larger version.)

Next item in to-do list was implementing a path selection panel in arm. After a thorough discussion on IRC with the core Tor developers it was decided that giving end-users complete access to the PathSupport component of TorCtl would ultimately hurt their anonymity with little (if any) pay-off. Letting them choose the country of their exit relay was still seen as a useful feature which is what I’m working on right now.

Meanwhile, while setting the configuration options needed to choose an exit country I figured that arm should be notified of any changes made by other controllers to the options in question. Damian pointed me to ticket #1692 which proposed an enhancement for this very purpose. He had already created a patch for the job but few quirks needed to be addressed before the patch was ready for being merged. Thanks to help from Nick Mathewson and Robert Ransom I was able to fix the code and get it inside the tor and tor-spec repositories.

To make use of the new CONF_CHANGED event I also wrote patches for TorCtl and Vidalia. The TorCtl patch shall help me in making arm aware of configuration changes. While the Vidalia one is a proof-of-concept which created a signal and connected it to a debug function which for the time being just logs the options changed for the running Tor instance.

I hope to be done with exit node selection within a couple of days after which I’ll port the feature to CLI version of arm. It’ll be a good little exercise to start decoupling the arm interfaces from its backend so that future development is more streamlined and fun.

Damian and I have decided that the GUI prototype for arm should be released to the public for a feedback round. As always, the code is available from the Git repository.

So far, the graphs and logs now support a look more conforming with the rest of the GTK+ theme:

(Click on the thumbnail for larger version.)

The connections panel lists circuits and control ports in use. The resolvers in use are in fact, the same ones used by the CLI so the results are pretty consistent:

(Click on the thumbnail for larger version.)

The “sticky” or “header” panel which provided general Tor/arm info has been moved to the notebook for giving it more horizontal space:

(Click on the thumbnail for larger version.)

Owing to my lack of creativity the GUI was named “garm”. Which coincidentally turned out to be the name of a talking dog from one of Tolkein’s short stories. Proving once again that Tolkein/Middle Earth/Norse mythology are any geek’s safe fallback plan for being imaginative.

While the GUI is still in infancy, we’d love any feedback we can get. If you have a suggestion you can post a comment here, drop by at #tor-dev or (preferably) mail your thoughts to tor-talk. I’ll be grateful for any ideas that end up making garm more useful for relay operators.

The great thing about a command line application is being able to SSH into the thing from anywhere and with anything. Nevertheless, the general public appeal of GUIs has always remained undeniable. After all, over the decades one of the favorite pass times of Steve Jobs — the man who knows a thing or two about public appeal — has been suing and/or getting sued for patents related to GUI. It’s not to say that we are planning an iTorMonitor for App Store (you still have iSSH if you’d like), but a graphical interface shall hopefully go a long way for attracting newbie relay operators.

The first items to be ported to GUI were the bandwidth graphs. After a thorough discussion on #tor-dev regarding how to achieve graphing with respect to feature sets, packaging issues and wheel reinvention; cagraph was chosen as the way to go (among Matplotlib and drawing directly to GDK surfaces). I took screenshots of both interfaces running side-by-side in order to judge how accurate the graphs were and the results look fine:

(Click on the thumbnails for larger version.)

Next up were the log messages dispatched by arm or Tor. While Damian would not be entirely happy with the fact that I’m not terribly innovative with the UI translation ;-) , I did stumble upon an interesting side-feature of using timestamp based sorting. The user can sort the entries in ascending order and he’ll always see the recent-most entry as it pops up in the view, or he can revert the order and see old entries at his leisure while the new entries populate elsewhere below.

(Click on the thumbnail for larger version.)

One other aspect I noticed while designing the UIs was that I have atrocious color selection skills. The color scheme of the entire application isn’t consistent and might even invite a backlash once it goes public. Therefore I plan on discarding all hardcoded colors in favor of theme colors from GTK+ itself — lest the GUI be packaged into a separate arm-trippy once it makes to major distros.

The first thing that comes to mind after seeing “curses” and “Python” in the same sentence is “go away or I shall taunt you a second time”. After spending a while trying to write text-mode interfaces, it only starts ringing truer.

Coding period for Google Summer of Code 2011 officially began last week. Because of exams and some subsequent issues involving my university I had been lagging behind my intended schedule. With help from Damian Johnson though I was able to get my feet wet quickly and start integrating menus in arm. Luckily, the arm codebase was very well-written and neatly organized which simplified my task and allowed me to end up with a functional implementation by the end of first week:

(Click on the thumbnail for larger version.)

The code can be accessed via my Git repository at Tor Project. In addition to that I also now own a shiny krkhan@torproject.org email address which is currently setup to forward messages to my primary mail.

Menus still needs a bit of polishing as the controls are not completely intuitive and I still need to bug-hunt thoroughly on varying screen sizes. For the time being they work well enough to control all aspects of arm except for quitting or resetting Tor, which I shall be fixing after figuring out a few quirks.

“As a child my family’s menu consisted of two choices: take it or leave it.” — Buddy Hackett

After completing my project for last year‘s Summer of Code my degree was delayed for a while because of flunking a few courses. Glass half full, this gave me the chance to participate once more before I finally graduate this summer — Deo volente.

In a repeat of last year I once again became a duplicate student as my proposals for both Fedora and Tor projects ended up in top slots. Both organizations were really nice in the de-duplication process and inquired about my preference which inclined towards Tor since it allowed me to contribute to fields of privacy and anonymity. In words of Free Software Foundation:

Using free software, Tor has enabled roughly 36 million people around the world to experience freedom of access and expression on the Internet while keeping them in control of their privacy and anonymity. Its network has proved pivotal in dissident movements in both Iran and more recently Egypt.

This by no means implies any lesser significance for Fedora in my life and I would have been as much enthused had it been my mentoring organization for the summer instead of EFF. However, since I had to make a choice I went for the option where I would be contributing more directly to privacy enhancing technologies.

My project this summer revolves around improving the excellent Anonymizing Relay Monitor (arm) which is used by the Tor community to monitor and control bridges and relays. The complete proposal can be read here (or via the original PDF). It contains nitty-gritty details of my summer aspirations as well as non-technical stuff like why I want to contribute to this particular field.

I Know What I Did Last Summer, and I’m looking forward to doing it again. Quoting myself from FLOSS Manuals:

“Summer of Code is about much more than just code. The sheer fun of integrating with the open-source community and your mentoring organization can in fact outweigh the gratification of actual coding. “

Smolt is a hardware profiler for Linux distributions which makes it easier for end-users to report back their machine configurations to a centralized database. Mike McGrath provides an excellent backend for developing Smolt GUIs which I have coupled with GTK+ for GSmolt:

(Click on the thumbnails for larger versions.)

The script can be found at the gsmolt repository on GitHub. Things on todo list include profile reporting in a separate thread and better error handling. I’ll provide RPM and Deb packages when the code is ready for a public release.

As a side note, this is the first project I have tracked using GitHub (as opposed to Launchpad + Bazaar). While Launchpad has its added advantage of PPAs which make it easier to push out public releases for Debian derivatives, I’m liking the Git experience so far. Hopefully some day Copr shall mature to a point where it can be the end-all, be-all Launchpad alternative for Fedora users.

The Program

And with the final code submission uploaded at Google Code, Summer of Code 2010 officially draws to a close. What a ride it has been!

My first post on Inspirated regarding Summer of Code took place all the way back in March 07 when I submitted a proposal for Fedora. Three and a half years later, I finally appreciate the fact that GSoC is about much more than just the code. In fact, “Summer of Open-Source” would probably be a more suitable title for the program despite sounding half as sexy. When my proposals got rejected I had to learn, contribute and integrate more with the open-source crowd. When one of the applications finally made it beyond the selection process, I had to “bond” with the organization I was going to be working with. During the coding weeks, I had to communicate regularly with my mentor and Ubuntu community in order to ensure that I was progressing in the right direction. To summarize, becoming a part of the open-source universe is as important an aspect of GSoC as producing open-source code.

The Code

Here’s a quick rundown of the code produced for the program (without including the intermediate branches and patches):

Attachment Search

• Merge Revision: Expot API call for listing the source packages a team has subscribed to.
• Branch: Search attachment files using Horspool’s algorithm. The branch eventually didn’t make it upstream as it was decided that with the existing implementation the searches would be too expensive for Launchpad, essentially requiring a solution with a different design approach.

Arsenal and python-launchpadlib-toolkit Modifications

• Package, Package, Package: These libraries had to be modified extensively in order to provide support for the Arsenal scripts developed during GSoC. Classes like LaunchpadApplication, LaunchpadBugzillaApplication and BugzillaAdapter perform regular chores for these applications e.g., authentication, web-scraping and keyring manipulation.

Attachment Upstreamer

• Script: Command-line script for upstreaming attachments from a Launchpad bug to a remote Bugzilla.
• Script, Template: CGI script for demonstrating the capabilities of Arsenal library and python-launchpadlib-toolkit.

Bug Matchmaker

“Incompatibility: In matrimony, a similarity of tastes, particularly the taste for domination.” — Ambrose Bierce

Fortunately, matchmaking across different bug trackers was easier.

• Script: Command-line script for searching a remote Bugzilla for similar entries to a given Launchpad bug.

Automatic Patcher

• Script: Command-line script for automatically generating patched Debian packages for a given Launchpad bug using diff files found in its attachments.

The Nutshell

The processes of collaborating, getting code reviewed and improving accordingly has gone a long way in instilling more confidence in me as a FLOSS developer. On the technological side of things I’ve had to deal with REST APIs, Zope Interfaces, OAuth procedures, web-scrapers based on Curl, Debian patch systems, a plethora of Python and a good-bit doze of different development paradigms.

It has been awesome working with the Ubuntu community and Bryce’s guidance has been absolutely crucial for ensuring that something useful was produced by my project. I’d like to thank everyone involved in the program for making it so much fun. I Know What I Did This Summer.

As eleven weeks of the-best-summer-ever draw to an end, here’s the final coding report for GSoC 2010.

Related Links

Time Spent

60 hours.

Highlights

The week was spent mostly cleaning and packaging the code accumulated over the summer. To demonstrate some of the aspects of the Arsenal library, I also created a proof-of-concept CGI script which upstreams Launchpad attachments for a bug to a remote Bugzilla. The task was fun, as the efforts put into refactoring things into launchpadlib-toolkit and BugzillaAdapter finally paid off and it took only a few hours to get the script working (that too with most of the time spent learning AJAX).

Concerns

None.

Waiting Items

None.

Stalled Items

None.

Accomplishments

• Branch, Merge Revision:
  • Revision: Added support for quilt.
  • Revision: Added support for using patch utility for quilt packages where the diff files update debian/* stuff themselves.
  • Revision: Cleaned up the library to provide LaunchpadApplication and LaunchpadBugzillaApplication.
  • Revision: Fixed BugPatcher to use LaunchpadApplication as base class.
  • Revision: Cleaned up LaunchpadBugzillaApplication to take username password as arguments instead of modifiers.
• Branch, Merge Revision: Fixed packaging issues to release debs for Karmic and Lucid.
• Branch: Implemented a CGI script demonstrating the upstreaming capabilities of Arsenal library. An example run can be seen in this screencast.

Minor Tasks

• Revision: Some more code cleanup.
• Revision: Check launchpadlib version before appending ‘/beta‘ during API URL detection.

Actions for the Following Report

• Fill the final evaluation.
• Write a summary of the overall GSoC experience.
• Start waiting for the t-shirt.

Related Links

Time Spent

90 hours.

Highlights

• Refactored Attachment Upstreamer in order to migrate Launchpad and Bugzilla chores to launchpadlib-toolkit and BugzillaAdapter.
• Implemented match-upstream.py for matchmaking Launchpad bugs in remote trackers.
• Implemented bug-patcher.py for generating patched Debian packages for Launchpad bugs.

Concerns

None.

Waiting Items

None.

Stalled Items

None.

Accomplishments

• Branch, Merge Revision: Migrated Attachment Upstreamer to use launchpadlib-toolkit.
• Branch, Merge Revision: Migrated Attachment Upstreamer to use BugzillaAdapter.
• Revision: Updated launchpadlib-toolkit to serve scripts such as Attachment Upstreamer through attachment filters and wrappers.
• Branch, Merge Revision: Implemented match-upstream.py with support for multiple level searches for finding a bug’s attributes in a remote tracker. Supports searching titles, git commit ids and attachment filenames.
• Branch: Implemented bug-patcher.py with support for modifying Debian packages which use cdbs patch system to generate a patched version using a bug’s attachments.

Minor Tasks

• Created a LaunchpadBugzillaApp class which shares majority of the initialization code for send-attachments-upstream.py, match-upstream.py and bug-patcher.py.
• Fixed various bugs in LaunchpadBugzillaApp for dealing with Gnome Keyring and Launchpad authentication.
• Added various filters to launchpadlib-toolkit.

Actions for the Following Report

• LaunchpadBugzillaApp should be derived from LaunchpadApp which would allow using the latter for scripts such as bug-patcher.py where Bugzilla portions aren’t required.
• Update Bug Patcher to include support for remaining Debian patch systems.
• Clean up the code accumulated over GSoC development and write tests.

Related Links

Time Spent

60 hours.

Highlights

New features were added to Attachment Upstreamer in order to make it more suitable for issues encountered by Ubuntu maintainers (as suggested by Bryce from his experience as the X.org maintainer).

Concerns

None.

Waiting Items

None.

Stalled Items

None.

Accomplishments

• Branch, Merge Revision: Implemented caching of Bugzilla credentials using Gnome Keyring and ConfigParser.
• Merge Revision:
  • Branch: Added support for excluding attachments based on filename matching using glob patterns.
  • Branch: Added support for extracting Tar and Zip archives when the number of files in them is below a specified limit.
  • Branch: Added support for excluding attachments based on their sizes, optionally Gzipping them in an effort to make the size acceptable.
  • Branch: Added support for enforcing content-types of attachments based on their filenames.

Minor Tasks

• Various bugfixes and code-cleanup for previously merged GSoC code.

Actions for the Following Report

The Launchpad and Bugzilla sides of the Upstreamer are to be cleaned up and made dependent on launchpadlib-toolkit and BugzillaAdapter respectively. This will help future scripts which rely on Bugzilla communication as well as make such things agnostic to the implementation lying beneath the adapter (e.g., whether we’re using Curl/XML-RPC/REST to talk to the server).