Inspirated

Here’s a quick little script which I wrote to tabulate the word frequencies in Pidgin logs. Simple, you point it towards a contact’s log directory:

$ ./purple-stats.py /home/krkhan/.purple/logs/msn/krkhan\@inspirated.com/some.friend\@some.gmail.com

And it gives you the words in their descending order of usage:

         0: you        (38)
         1: it         (30)
         2: to         (24)
         3: the        (22)
         4: in         (22)
         5: lol        (22)
         6: of         (22)
         7: so         (18)
         8: is         (18)
         9: what       (16)
         ...

As usual, Python was used for the dirty work:

purple-stats.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

#!/usr/bin/env python
 
from operator import itemgetter
from string import punctuation
 
import locale
import os
import sys
 
from BeautifulSoup import BeautifulSoup
 
if __name__ == "__main__":
    if len(sys.argv) < 2:
        print "usage:", sys.argv[0], "<logs directory>"
    dir = sys.argv[1]
 
    contents = filter(lambda x: x[-5:] == '.html', os.listdir(dir))
    stats = {}
    for entry in contents:
        path = os.path.join(dir, entry)
        with open(path, 'r') as fd:
            data = fd.read()
        soup = BeautifulSoup(data,
            convertEntities=BeautifulSoup.ALL_ENTITIES)
        spans = soup.findAll('span')
        for span in spans:
            for word in span.text.split():
                word = word.strip(punctuation).lower()
                if len(word) < 2:
                    continue
                stats[word] = stats.get(word, 0) + 1
 
    sorted_stats = sorted(stats.iteritems(), key=itemgetter(1))
    sorted_stats.reverse()
    for num, (word, count) in enumerate(sorted_stats):
        line = "%10d: %-10s (%d)" % (num, word, count)
        line = line.encode(locale.getpreferredencoding())
        print line

Yesterday I needed to copy the MBR of a drive over another. Googling a little I found the following command in various tutorials:

-bash-$ dd if=/dev/sda of=/dev/sdb bs=512 count=1

Where /dev/sda and /dev/sda were the original and target hard disks respectively. The command did complete its work in a snap but it also made me learn a thing about MBR structures the hard way: Only 446 bytes of the MBR contain boot code, the next 64 contain the partition table!

The implications of the lesson being, if partition tables of both hard disks differ — which unfortunately was the case with me — the partition table of the target hard-disk will be overwritten. The correct way would therefore be:

-bash-$ dd if=/dev/sda of=/dev/sdb bs=446 count=1

In case you did mess up the table, I recommend TestDisk for recovering your partitions.

A match made in heaven.

Using IRC in a reliable way has turned out to be kind of a challenge for me in the past couple of months. I had my cellphone connected 24/7 to the IRC channels I needed to idle in. Unfortunately, I don’t live in a 3G country so any voice-calls interrupted the whole thing.

I also wanted to use my laptop for IRC-ing whenever I was at home. But the inconsistent internet connection didn’t make thing any easier. There was always this lingering fear of missing important messages during one of the disconnections. I couldn’t foresee a solution which would fix all of the mentioned issues until someone recommended using Irssi along with GNU Screen.

This not only fixed every little issue I had ever had with IRC but also made full utilization of my love for all things command-line. In summary: I now have this “permanent” IRC session running at an SSH server in Lithuania. Whenever I feel like it, I can “attach” my laptop or my mobile and start using Irssi. If I receive any voice-calls, the IRC session still continues to work and I can “reattach” later on. Simply put, the session continues running even when I’m not attached to it through either device and at any time of the day I can simply connect to it and resume working through that day’s IRC activity.

Here’s a screenshot that shows me connected to the #gsoc channel on Freenode first on my laptop and then on my E71 using the same screen session:

PuTTY for Symbian is used for SSH-ing on the Nokia phone. If I ever rank top 5 situations where CLI absolutely pwns GUI in terms of efficiency and usage, this nifty setup is definitely going to make the list.

While I would have sold my clothes, PS2 as well as my soul to get my hands on a N900, I couldn’t strike a practical deal and in the end had to settle for E71. Now, while E71 does not ooze s** by running a full-fledged Linux distro it still does a pretty good job of running Symbian. The specs are fun, the keypad is great and the build quality is nothing but ergonomic love.

After completing 71 days of tinkering with the new phone, I have settled on the choice of apps which would become an integral part of my digital life. I’m listing the top ten here, but the arrangement is in no way meant to be indicative as an absolute index of application quality. These are just the ones that I found useful.

10. ScreenSnap — Honorary Mention

It would be a bit unfair to not list the very application that powers the visual aspects of this post. So here we are with the self-referential snap:

9. SmsOne — Hassle-Free Flash SMS

“Flash” texts — the ones which always instantly show up on the recipient’s cell and are not saved in the inbox unless specifically made to — have always been a speciality of applications with poor interfaces as well as poorer integration with the rest of the phone. SmsOne, on the other hand, has a simple interface with excellent phonebook integration for making the job as simple as possible.

8. Mobipocket MobiReader — eBooks Made Fun

Reading a large PDF file on your E71 is indeed possible, but it takes away any fun whatsoever by forcing you to deal with redundant scrolling and weird font sizes. MobiReader combined with the desktop software Calibre is the ultimate eBook reading experience on Symbian phones.

7. Gravity — Pulling Tweets Consistently

The one word for summarizing Gravity would be: slick. Of all the Twitter applications for Symbian — and there are quite a lot — Gravity is the most organized, reliable and pretty-looking competitor around.

6. PuTTY — That Insane Power in Your Hands

I’m lying down in my bedroom and I want to play this song on the speakers in my drawing room. I’m in my university and I want to start this download on my laptop which is unfortunately back at home. I’m in another city and I want to see the contents of this file which is in my home PC.

I can. Because I have that command-line access at my fingertips:

5. mIRGGI — IRC Never Dies

Open-source — being the distributive collaboration that it is — depends heavily on mailing lists and IRC channels for communication. For the latter, mIRGGI works flawlessly. In fact, you can even do fancy stuff like using different network connections for different channels.

4. Python — “It’s …”

For all the comparisons made over the years between Perl and line-noise, Symbian C++ easily beats both by a mile. Which is why your choices are rather limited if you want to program for Nokia phones while preserving your sanity at the same time. Your best bet is to go for Maemo. Unfortunately, that ain’t exactly an affordable option for everyone. The other way is to use PyS60 rely on that programming language which makes every other alternative pine for the fjords.

3. MSDict Viewer — Dictionaries which Help

A dictionary is one of the least useful things you can carry around. Even with one of them installed on your phone, they rarely get used. Similarly, the pronunciation guides used in those dictionaries are cryptic symbols at best. Add the ability of audio pronunciations and you suddenly have a lovely app referring which almost becomes an addiction.

2. Google Maps — Never Lose Yourself

For people who’re not especially good with directions, there couldn’t have ever been a better solution:

1. Opera Mobile — Synonym for Excellence

Even if this list is tailored more to my requirement, get some hundred people to make similar ones and Opera will be the topper in majority of the rankings. Opera bridges that gap between mobile browsing and real browsing and is one of those applications which epitomize how software should be engineered.

The standard way of including source code listings in Beamer is to use the semiverbatim environment. Needless to say, it does not provide all the syntax highlighting and line-numbering love of the listings package. Combine the two and you have something pretty as well as extremely helpful in delivering presentations which have code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

\documentclass{beamer}
 
\setbeamercovered{transparent}
 
\usepackage{pxfonts}
\usepackage{listings}
 
\begin{document}
 
\lstset{language=python,
        numbers=left,
        numberstyle=\tiny,
        showstringspaces=false,
        aboveskip=-40pt,
        frame=leftline
        }
 
\begin{frame}[fragile]
\frametitle{\texttt{parrot.py}}
\begin{semiverbatim}
\pause
\begin{lstlisting}
if __name__ == "__main":
 
\end{lstlisting}
 
\pause
\begin{lstlisting}[firstnumber=last]
    print "Oh yes, the, uh,
           the Norwegian Blue..."
    print "What's, uh...
           What's wrong with it? "
 
\end{lstlisting}
 
\pause
\begin{lstlisting}[firstnumber=last]
    print "I'll tell you what's wrong with it,
           my lad."
    print "'E's dead,
           that's what's wrong with it!"
 
\end{lstlisting}
\end{semiverbatim}
\end{frame}
 
\end{document}

Thanks to Christoph Korn, Ubuntu users can now install the package with a single click from the GetDeb repository. The Deb file itself is available on the release page here, along with an RPM for Fedora users.

The looks:

And the hooks:

Changelog:

• Fixed:
  • Bug #522735: Facebook: Application Request Limit Reached
  • Bug #523378: Connection reset by peer
  • Bug #522487: Facebook Friends Graph fails when friends have a dash in their name [patch by Little Jawa]

Threads are love. Threads are speed. And more often than not, threads are a consistent PITA. However, I’ve had an accidental epiphany just a few hours ago:

“When in doubt When you need to communicate among threads, use synchronized Queues.”

There. This magic mantra will solve more issues in your life than you can ever imagine, and certainly more than I expected.

Getting back to the topic at hand, adding threading support to the program has sped up the bookmark checking process by a factor of about 435895234. Coupled with fixing of some parsing bugs, Bookmark Undertaker v0.3 is finally capable of providing a quick, stable and consistent way of sanitizing your Firefox favorites:

This time, I’ve also tried to provide Deb and RPM packages on the release page for easy installation by the Debian/Ubuntu/Fedora populace.

Ushering in the era of communist applications:

“If everyone gives one thread, the poor person will have a shirt.” — Russian Proverb

I never really thought anyone other than me would be interested in seeing gargantuan graphs of their friends’ connections until I found out through this post on the OMG! Ubuntu! blog that my application was included in the GetDeb repository for Ubuntu users. I have not used Ubuntu myself since about never, but apparently you can now install the application on Karmic Koala with just a few clicks.

Edit: I have now tested the installation on Karmic myself and can guarantee that it indeed works without any fuss. Gotta love Launchpad/Ubuntu.

The application itself was in a pretty much skeletal state of being so I was a little taken aback by the exposure. Nevertheless, I was reminded of the famous aphorism apropos of open source development:

“Release early, release often.” — Linus Torvalds

And indeed, the bug reports that came from users were a valuable byproduct of the Ubuntu push as I had stopped development on the script after it started working fine for me.

“Anyone who slaps a ‘this page is best viewed with Browser X’ label on a Web page appears to be yearning for the bad old days, before the Web, when you had very little chance of reading a document written on another computer, another word processor, or another network.” — Tim Berners-Lee in Technology Review, July 1996

People never learn. Slapping such labels is one thing, they even go as far as adopting brain-dead practices of checking user-agent strings and refusing service to any browser not originating from Redmond. For example, Opera Mobile — the sexiest mobile application on planet — works astonishingly well for Javascript websites. Nevertheless, when trying to browse my university’s academic management portal on it I am presented with a big ugly “We’re dumb, you need to open this page with Internet Explorer 6.0 or later because it uses JAVASCRIPTXX0RZ.” Even though Opera does allow spoofing of user-agent strings, the fake strings still contained “Symbian” as the operating system which still resulted in incompatibility errors.

As ever, Python came to the rescue. Firing up the Twisted framework, I created a simple HTTP proxy which modifies the user-agent string on the fly. Peaches:

The script is still very quirky and is the farthest thing from what you’d call a stable solution. You can download the inital release here. The zip file contains the tiny proxy server script as well as Twisted and Zope dependencies. Good luck with trying to counter retards who’re doing everything they can to avoid compatibility. Yes, even 14 years after Sir Tim’s veracious proclamation.

Continuing the migration to E71, here’s the new release for Inbox Stats which works on Python 2.5 releases:

inboxstats-1.1.zip