Inspirated

Smolt is a hardware profiler for Linux distributions which makes it easier for end-users to report back their machine configurations to a centralized database. Mike McGrath provides an excellent backend for developing Smolt GUIs which I have coupled with GTK+ for GSmolt:

(Click on the thumbnails for larger versions.)

The script can be found at the gsmolt repository on GitHub. Things on todo list include profile reporting in a separate thread and better error handling. I’ll provide RPM and Deb packages when the code is ready for a public release.

As a side note, this is the first project I have tracked using GitHub (as opposed to Launchpad + Bazaar). While Launchpad has its added advantage of PPAs which make it easier to push out public releases for Debian derivatives, I’m liking the Git experience so far. Hopefully some day Copr shall mature to a point where it can be the end-all, be-all Launchpad alternative for Fedora users.

Screensavers like glmatrix have long been used by *nixers to woo people by showing them customizable animations as desktop wallpapers. Users of desktop environments such as Xfce have to however use xwininfo to determine and use the window IDs of their desktops (as the “-root” option stops working when the root window is overlayed by respective desktop managers e.g., Xfdesktop). For those who want to automate the startup process of XScreenSaver wallpapers in such environments, here’s a quick command you can use:

$ /usr/libexec/xscreensaver/glmatrix -window-id $(xwininfo -name "Desktop" | grep 'Window id' | sed 's/.*\(0x[0-9a-z]*\).*/\1/g')

Results:

(Click on the thumbnails for larger versions.)

The final package arrived today containing the (100% Organic O*******) shirt and certificate:

(Click on the thumbnail for larger version.)

The Program

And with the final code submission uploaded at Google Code, Summer of Code 2010 officially draws to a close. What a ride it has been!

My first post on Inspirated regarding Summer of Code took place all the way back in March 07 when I submitted a proposal for Fedora. Three and a half years later, I finally appreciate the fact that GSoC is about much more than just the code. In fact, “Summer of Open-Source” would probably be a more suitable title for the program despite sounding half as sexy. When my proposals got rejected I had to learn, contribute and integrate more with the open-source crowd. When one of the applications finally made it beyond the selection process, I had to “bond” with the organization I was going to be working with. During the coding weeks, I had to communicate regularly with my mentor and Ubuntu community in order to ensure that I was progressing in the right direction. To summarize, becoming a part of the open-source universe is as important an aspect of GSoC as producing open-source code.

The Code

Here’s a quick rundown of the code produced for the program (without including the intermediate branches and patches):

Attachment Search

• Merge Revision: Expot API call for listing the source packages a team has subscribed to.
• Branch: Search attachment files using Horspool’s algorithm. The branch eventually didn’t make it upstream as it was decided that with the existing implementation the searches would be too expensive for Launchpad, essentially requiring a solution with a different design approach.

Arsenal and python-launchpadlib-toolkit Modifications

• Package, Package, Package: These libraries had to be modified extensively in order to provide support for the Arsenal scripts developed during GSoC. Classes like LaunchpadApplication, LaunchpadBugzillaApplication and BugzillaAdapter perform regular chores for these applications e.g., authentication, web-scraping and keyring manipulation.

Attachment Upstreamer

• Script: Command-line script for upstreaming attachments from a Launchpad bug to a remote Bugzilla.
• Script, Template: CGI script for demonstrating the capabilities of Arsenal library and python-launchpadlib-toolkit.

Bug Matchmaker

“Incompatibility: In matrimony, a similarity of tastes, particularly the taste for domination.” — Ambrose Bierce

Fortunately, matchmaking across different bug trackers was easier.

• Script: Command-line script for searching a remote Bugzilla for similar entries to a given Launchpad bug.

Automatic Patcher

• Script: Command-line script for automatically generating patched Debian packages for a given Launchpad bug using diff files found in its attachments.

The Nutshell

The processes of collaborating, getting code reviewed and improving accordingly has gone a long way in instilling more confidence in me as a FLOSS developer. On the technological side of things I’ve had to deal with REST APIs, Zope Interfaces, OAuth procedures, web-scrapers based on Curl, Debian patch systems, a plethora of Python and a good-bit doze of different development paradigms.

It has been awesome working with the Ubuntu community and Bryce’s guidance has been absolutely crucial for ensuring that something useful was produced by my project. I’d like to thank everyone involved in the program for making it so much fun. I Know What I Did This Summer.

As eleven weeks of the-best-summer-ever draw to an end, here’s the final coding report for GSoC 2010.

Related Links

Time Spent

60 hours.

Highlights

The week was spent mostly cleaning and packaging the code accumulated over the summer. To demonstrate some of the aspects of the Arsenal library, I also created a proof-of-concept CGI script which upstreams Launchpad attachments for a bug to a remote Bugzilla. The task was fun, as the efforts put into refactoring things into launchpadlib-toolkit and BugzillaAdapter finally paid off and it took only a few hours to get the script working (that too with most of the time spent learning AJAX).

Concerns

None.

Waiting Items

None.

Stalled Items

None.

Accomplishments

• Branch, Merge Revision:
  • Revision: Added support for quilt.
  • Revision: Added support for using patch utility for quilt packages where the diff files update debian/* stuff themselves.
  • Revision: Cleaned up the library to provide LaunchpadApplication and LaunchpadBugzillaApplication.
  • Revision: Fixed BugPatcher to use LaunchpadApplication as base class.
  • Revision: Cleaned up LaunchpadBugzillaApplication to take username password as arguments instead of modifiers.
• Branch, Merge Revision: Fixed packaging issues to release debs for Karmic and Lucid.
• Branch: Implemented a CGI script demonstrating the upstreaming capabilities of Arsenal library. An example run can be seen in this screencast.

Minor Tasks

• Revision: Some more code cleanup.
• Revision: Check launchpadlib version before appending ‘/beta‘ during API URL detection.

Actions for the Following Report

• Fill the final evaluation.
• Write a summary of the overall GSoC experience.
• Start waiting for the t-shirt.

Related Links

Time Spent

90 hours.

Highlights

• Refactored Attachment Upstreamer in order to migrate Launchpad and Bugzilla chores to launchpadlib-toolkit and BugzillaAdapter.
• Implemented match-upstream.py for matchmaking Launchpad bugs in remote trackers.
• Implemented bug-patcher.py for generating patched Debian packages for Launchpad bugs.

Concerns

None.

Waiting Items

None.

Stalled Items

None.

Accomplishments

• Branch, Merge Revision: Migrated Attachment Upstreamer to use launchpadlib-toolkit.
• Branch, Merge Revision: Migrated Attachment Upstreamer to use BugzillaAdapter.
• Revision: Updated launchpadlib-toolkit to serve scripts such as Attachment Upstreamer through attachment filters and wrappers.
• Branch, Merge Revision: Implemented match-upstream.py with support for multiple level searches for finding a bug’s attributes in a remote tracker. Supports searching titles, git commit ids and attachment filenames.
• Branch: Implemented bug-patcher.py with support for modifying Debian packages which use cdbs patch system to generate a patched version using a bug’s attachments.

Minor Tasks

• Created a LaunchpadBugzillaApp class which shares majority of the initialization code for send-attachments-upstream.py, match-upstream.py and bug-patcher.py.
• Fixed various bugs in LaunchpadBugzillaApp for dealing with Gnome Keyring and Launchpad authentication.
• Added various filters to launchpadlib-toolkit.

Actions for the Following Report

• LaunchpadBugzillaApp should be derived from LaunchpadApp which would allow using the latter for scripts such as bug-patcher.py where Bugzilla portions aren’t required.
• Update Bug Patcher to include support for remaining Debian patch systems.
• Clean up the code accumulated over GSoC development and write tests.

Related Links

Time Spent

60 hours.

Highlights

New features were added to Attachment Upstreamer in order to make it more suitable for issues encountered by Ubuntu maintainers (as suggested by Bryce from his experience as the X.org maintainer).

Concerns

None.

Waiting Items

None.

Stalled Items

None.

Accomplishments

• Branch, Merge Revision: Implemented caching of Bugzilla credentials using Gnome Keyring and ConfigParser.
• Merge Revision:
  • Branch: Added support for excluding attachments based on filename matching using glob patterns.
  • Branch: Added support for extracting Tar and Zip archives when the number of files in them is below a specified limit.
  • Branch: Added support for excluding attachments based on their sizes, optionally Gzipping them in an effort to make the size acceptable.
  • Branch: Added support for enforcing content-types of attachments based on their filenames.

Minor Tasks

• Various bugfixes and code-cleanup for previously merged GSoC code.

Actions for the Following Report

The Launchpad and Bugzilla sides of the Upstreamer are to be cleaned up and made dependent on launchpadlib-toolkit and BugzillaAdapter respectively. This will help future scripts which rely on Bugzilla communication as well as make such things agnostic to the implementation lying beneath the adapter (e.g., whether we’re using Curl/XML-RPC/REST to talk to the server).

Firefox 4 offers some compelling features such as HTML 5 improvements and a new add-on manager. Since it’s quite painstaking to compile the beta from source and quite messy to place pre-compiled binaries in system default folders (not to mention the compatibility checks and upgrade chores that would interrupt at each launch if you go back and forth between different versions), I created a repository at repo.inspirated.com which can be used to test the beta version without touching any 3.6 stable release already installed on the system:

To use the repository, issue the following commands:

$ su -c 'wget https://repo.inspirated.com/inspirated.repo -O /etc/yum.repos.d/inspirated.repo'
$ su -c 'yum install firefox-beta'

The beta refuses to run if an instance of old Firefox is already active. Therefore, close the older Firefox and then launch the 4.0b1 version using firefox-beta command or the “Firefox Beta” shortcut in the applications menu. A new profile shall be created at ~/.mozilla/firefox/beta/ in order to leave your older profile’s settings, bookmarks and extensions etc. intact.

(Click on the thumbnail for larger version.)

Related Links

Time Spent

50 hours.

Highlights

The send-attachments-upstream.py script was migrated from XML-RPC to Curl for communicating with Bugzilla. The script was the refactored in order to provide capabilities such as attachment filtering. Various bugfixes and improvements were catered to along the way.

Concerns

None.

Waiting Items

None.

Stalled Items

None.

Accomplishments

• Branch, Merge Revision: Reimplemented attachment sending using pycurl.
• Branch, Merge Revision: Refactored the script in order to provide options such as -o (copy only attachments uploaded by bug owner).

Minor Tasks

• Branch, Merge Revision: Fixed regular expressions for parsing results and handling of Unicode attachment titles.

Actions for the Following Report

Implement the following improvements in send-attachments.py:

• Caching Bugzilla credentials.
• Filename exclusion for attachments.
• Archive extraction.
• File size limits.
• A command-line switch to enforce MIME content-types based on file extensions.

Related Links

Time Spent

10 hours.

Highlights

Communicating with Bugzilla is done through the python-bugzilla wrapper library. This could have been achieved by using xmlrpclib directly but doing that would require reinventing a whole lot of wheels by handling Bugzilla specific XML-RPC eccentricities.

Concerns

None.

Waiting Items

None.

Stalled Items

None.

Accomplishments

• Branch: Added support for copying attachments to a remote bugzilla:

  $ ./send-attachments-upstream.py --user=krkhan@inspirated.com --pass=xxx https://bugs.launchpad.net/ubuntu/+bug/223435 https://partner-bugzilla.redhat.com/show_bug.cgi?id=593603

  $ ./send-attachments-upstream.py --user=krkhan@inspirated.com --pass=xxx https://bugs.launchpad.net/ubuntu/+bug/223435 https://partner-bugzilla.redhat.com/show_bug.cgi?id=593603

  Logging in Launchpad [Success <Logged in as Kamran Riaz Khan>] Logging in Bugzilla [Success <Logged in as krkhan@inspirated.com>] Uploading: Dependencies.txt [Success] Uploading: Disassembly.txt [Success] Uploading: ProcMaps.txt [Success] Uploading: ProcStatus.txt [Success] Uploading: Registers.txt [Success] Uploading: Stacktrace.txt [Success] Uploading: ThreadStacktrace.txt [Success] Uploading: Stacktrace.txt (retraced) [Success] Uploading: ThreadStacktrace.txt (retraced) [Success]

  Logging in Launchpad [Success <Logged in as Kamran Riaz Khan>] Logging in Bugzilla [Success <Logged in as krkhan@inspirated.com>] Uploading: Dependencies.txt [Success] Uploading: Disassembly.txt [Success] Uploading: ProcMaps.txt [Success] Uploading: ProcStatus.txt [Success] Uploading: Registers.txt [Success] Uploading: Stacktrace.txt [Success] Uploading: ThreadStacktrace.txt [Success] Uploading: Stacktrace.txt (retraced) [Success] Uploading: ThreadStacktrace.txt (retraced) [Success]

Minor Tasks

• Revision: Added python-bugzilla in lib and modified setup.py accordingly.
• Revision: Initial commit for sending attachments to a remote Bugzilla.
• Revision: Added error handling for API calls.

Actions for the Following Report

Add support for creating new bugs in a remote Bugzilla based on data from a Launchpad bug.