Inspirated

As I promised in the comments section of previous post, I set out on the adventure of recompiling Bro for Lantiq routers. As a result of the exercise I have new-found respect for open-source package maintainers. Holy waffles if troubleshooting build errors in a large Autotools mess isn’t the most hemorrhage-inducing activity known to mankind.

Anyways, this time I’ve tried to keep track of the changes I’ve been making along the way. The full set of updated Makefiles and patches is maintained in the openwrt-bro repo. Also, the compiled ipk packages for Atheros and Lantiq routers are available on the release page.

Now that I have a reasonably updated Buildroot on my system and an organized set of patches, feel free to request an ipk package for your router. While I can’t guarantee that the clusterfuck of patches will compile smoothly for your platform, I’ll still give it a try.

After posting the Bro port for OpenWRT on my blog roughly two years ago, I didn’t realize some people were already actually using it on their routers.

I had created an updated version of the port which I hadn’t posted on the blog. Digging in my archived files I finally found it today along with its sources:

• openwrt-bro-2.3.1.tar.gz
• bro_2.3.1-1_ar71xx.ipk

Word of caution though, my notes indicate that one of the default scripts was leaking memory and I never got around to figuring out which one. The workaround was to launch Bro in barebone mode with -b switch, which would prevent loading of default scripts.

# cat test.bro

event bro_init()
{
	print "Hello World!";
}

event new_connection(c: connection)
{
	print "New connection created";
}

# bro test.bro

Hello World!

# bro -C -b -i br-lan test.bro

Hello World!
New connection created
New connection created

If someone has cycles to spend and figure out which default script is leaking memory we can update the package to address the bug.

The paper chronicling our adventures with Bro IDS on home routers just got published in the latest issue of SIGCOMM CCR. Here’re the details:

Title: Rapid and Scalable ISP Service Delivery through a Programmable MiddleBox

Abstract: With only access billing no longer ensuring profits, an ISP’s growth now relies on rolling out new and differentiated services. However, ISPs currently do not have a well-defined architecture for rapid, cost-effective, and scalable dissemination of new services. We present iSDF, a new SDN-enabled framework that can meet an ISP’s service delivery constraints concerning cost, scalability, deployment flexibility, and operational ease. We show that meeting these constraints necessitates an SDN philosophy for a centralized management plane, a decoupled (from data) control plane, and a programmable data plane at customer premises. We present an ISP service delivery framework (iSDF) that provides ISPs a domain-specific API for network function virtualization by leveraging a programmable middlebox built from commodity home-routers. It also includes an application server to disseminate, configure, and update ISP services. We develop and report results for three diverse ISP applications that demonstrate the practicality and flexibility of iSDF, namely distributed VPN (control plane decisions), pay-per-site (rapid deployment), and BitTorrent blocking (data plane processing).

Published in: ACM SIGCOMM Computer Communication Review (Volume 44 Issue 3, July 2014)

Combined with the paper in IEEE COMST about botnet detection that was published last year, this yields a grand-total of 2 publications more than I thought would ever bear my name. In any case, my former colleagues are continuing their excellent work on the project which can be tracked at the iSDF wiki-page.

Recently I’ve been reading through these excellent books in my spare time:

• Understanding Cryptography by Christof Paar
• Practical Common Lisp by Peter Seibel (also available as a free eBook)

To ramp-up on both subjects with one shot I wrote an implementation of Elliptic Curve Crypto in Lisp. So far, it does EC versions of Diffie-Hellman, ElGamal and DSA. Some rudimentary testing was performed using the NIST-P192 curve and its corresponding ECDSA test vectors.

The package is available at GitHub in the krkhan/cl-ecc repository. Here’s a quick snippet of what the code looks like:

(defconstant *p17-curve*
  (make-instance
    'Curve
    :a 2
    :b 2
    :p 17
    :g (make-instance
         'Point
         :x 5
         :y 1)
    :n 19))

And an ECDSA with this curve:

(def-positive-test test-ecdsa ()
  (let* ((c *p17-curve*)
         (bob-priv 3)
         (bob-pub (ecdh-gen-pub c bob-priv))
         (msghash 8)
         (k 7)
         (sig (ecdsa-gen-sig c msghash bob-priv k)))
    (assert (sig-equalp sig (make-instance 'ECDSASig :r 0 :s 12)))
    (ecdsa-verify-sig c msghash sig bob-pub)))

As a disclaimer — even though I know no one would be stupid enough to do so — please do not use this code in a production environment. It was written for recreational purposes by a hobbyist who is bad with cryptography and even worse with Lisp. On the other hand, if you have any suggestions/patches, feel free to create an issue/pull-request on GitHub.

I needed to block some flows on OpenWRT from the Bro IDS. One option was to install the recent module for expiring iptables rules which sounded like an overkill. After some tinkering around I landed on using bash and at to expire the firewall rules after timeouts (luckily the at daemon was available on OpenWRT which made my job easier).

There are three parts to the process:

The bash script

First, a script which:

1. Constructs and adds the iptables rule to the FORWARD chain.
2. Constructs the corresponding deletion rule.
3. Creates a temporary bash script, writes the rule to it, makes the new script self-deleting.
4. Schedules a launch of the temporary script with at command.

Here’s the script:

#!/bin/sh
 
if [ $# -le 5 ] ; then
  echo "usage: $0 proto src sport dst dport timeout"
  exit 1
fi
 
proto=$1
src=$2
sport=$3
dest=$4
dport=$5
timeout=$6
 
echo "  proto: $1"
echo "    src: $2"
echo "  sport: $3"
echo "   dest: $4"
echo "  dport: $5"
echo "timeout: $6"
 
rule=""
 
if [ "$proto" != "any" ]; then
  rule="$rule --protocol $proto"
fi
 
if [ "$src" != "0.0.0.0" ]; then
  rule="$rule --source $src"
fi
 
if [ "$sport" != "0" ]; then
  rule="$rule --sport $sport"
fi
 
if [ "$dest" != "0.0.0.0" ]; then
  rule="$rule --destination $dest"
fi
 
if [ "$dport" != "0" ]; then
  rule="$rule --dport $dport"
fi
 
rule="$rule -j DROP"
 
echo "rule: $rule"
 
addcmd="iptables -I FORWARD $rule"
delcmd="iptables -D FORWARD $rule"
 
delscript=`mktemp`
echo "delscript: $delscript"
 
echo "#!/bin/sh" >>$delscript
echo $delcmd >>$delscript
echo "rm \"${delscript}\"" >>$delscript
chmod 755 $delscript
 
echo "adding iptable rule:"
echo $addcmd
`$addcmd`
 
atcmd="at -M -f $delscript now + $timeout min"
echo "creating at job for deletion:"
echo $atcmd
`$atcmd`

Given below is an example run. First, let’s print the default FORWARD chain:

# iptables -nL FORWARD

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             10.42.0.0/24         state RELATED,ESTABLISHED
ACCEPT     all  --  10.42.0.0/24         anywhere            
ACCEPT     all  --  anywhere             anywhere            
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Block a flow for 2 minutes:

# sh blockflow.sh tcp 50.50.50.50 50 60.60.60.60 60 2

  proto: tcp
    src: 50.50.50.50
  sport: 50
   dest: 60.60.60.60
  dport: 60
timeout: 2
rule:  --protocol tcp --source 50.50.50.50 --sport 50 --destination 60.60.60.60 --dport 60 -j DROP
delscript: /tmp/tmp.SAREJvtsK0
adding iptable rule:
iptables -I FORWARD --protocol tcp --source 50.50.50.50 --sport 50 --destination 60.60.60.60 --dport 60 -j DROP
creating at job for deletion:
at -M -f /tmp/tmp.SAREJvtsK0 now + 2 min
job 79 at Sun Jun 30 14:37:00 2013

Let’s check if the new rule was added:

# iptables -nL FORWARD

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
DROP       tcp  --  50.50.50.50          60.60.60.60          tcp spt:50 dpt:60
ACCEPT     all  --  anywhere             10.42.0.0/24         state RELATED,ESTABLISHED
ACCEPT     all  --  10.42.0.0/24         anywhere            
ACCEPT     all  --  anywhere             anywhere            
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

After 2 minutes, the temporary bash script shall remove the rule and then delete itself. To confirm:

# iptables -nL FORWARD

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             10.42.0.0/24         state RELATED,ESTABLISHED
ACCEPT     all  --  10.42.0.0/24         anywhere            
ACCEPT     all  --  anywhere             anywhere            
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

The Bro module

A simple module which exports just one function, i.e., BlockFlow::block which takes a conn_id and a count and calls the bash script with appropriate parameters:

module BlockFlow;
 
export {
  global block: function(id: conn_id, t: count);
}
 
function block(id: conn_id, t: count)
{
  print fmt("blocking %s:%d -> %s:%d for %d minutes", id$orig_h, id$orig_p, id$resp_h, id$resp_p, t);
 
  local protocol = get_port_transport_proto(id$resp_p);
  print fmt("protocol is: %s", protocol);
 
  local cmd: string = fmt("sh blockflow.sh %s %s %d %s %d %d", protocol
                                                             , id$orig_h, id$orig_p
                                                             , id$resp_h, id$resp_p, t);
  print fmt("executing: %s", cmd);
  system(cmd);
}

Bro module usage

And finally, using the module from a Bro script:

@load ./blockflow
 
event bro_init()
  {
    local id: conn_id;
    id$orig_h = 10.10.10.10;
    id$orig_p = 10/tcp;
    id$resp_h = 20.20.20.20;
    id$resp_p = 20/tcp;
    BlockFlow::block(id, 2);
  }

And the flow will be blocked for 2 minutes. Unfortunately, due to the way at command works the granularity of timeouts is limited to minutes. If you really want to block flows for only a few seconds a quick solution would be to use sleep in place of at before expiring the rule.

While I was at SysNet, we had been working on a project we called “Shrimp” — Software-defined Home Router Intelligent Monitoring Point. The goal of the project was to provide a framework for easy programmatic access to network monitoring on low-cost, commodity, home router devices. One of the requirements was to have an IDS on the home routers for which we chose Bro — the leading framework for semantic analysis of network traffic.

The OpenWRT OS was chosen as the target platform. Its SDK contained a cross-compile toolchain for CMake projects. However, during the compilation Bro tried to run the binpac and bifcl executables for processing intermediate files. The executables refused to run on the build platform if the target platform architecture was different (mostly the case, e.g., we were building on x86-64 and target was arm).

The (not-so-pretty ™) workaround we used was to build Bro twice. Once for the host, and once for the target. The CMake files were then patched to first generate binpac and bifcl binaries if they weren’t provided and then use the provided binaries if they were defined at make time. The first compile generated the binaries on x86-64 and the second compile (for arm) used the earlier binaries to process the bif files.

The Makefile and patches are available in this tarball: openwrt-bro.tar.gz, while the compiled ipk package is also available for installation. Here is a test execution of Bro on OpenWRT:

# bro –v

bro version 2.0

# cat test.bro

event bro_init()
{
	print "Hello World!";
}

event new_connection(c: connection)
{
	print "New connection created";
}

# bro test.bro

Hello World!

# bro -i br-lan test.bro

Hello World!
New connection created
New connection created

# ls

conn.log           notice_policy.log  reporter.log       weird.log
dns.log            packet_filter.log  test.bro

A heap of thanks to Zaafar for dealing with my messy code and providing the links to hosted files :) !

During the course of my work on botnet security we have had to deal with mammoth traffic traces captured at a local ISP. While analyzing the traffic we needed to extract traffic for some certain hosts out of large pcap files. An obvious solution would be to run tshark once for each host, filtering the traffic for that particular IP and writing it to a separate pcap file. However with the number of hosts approaching thousands and the pcap traces approaching terabytes in size tshark didn’t really fit the bill.

Initially I thought of writing a splitter in Python but my colleague’s aversion for using Python on large network traces coupled with lack of maintenance of libpcap bindings resulted in me going for C/libpcap directly. The new C-based slicer is available at our GitHub respository. It needs glib to compile though, as I needed a hash table implementation for maintaining the list of hosts that need to be sliced. The Makefile in the repository should take care of compiling with the appropriate flags.

Onto the performance, the speed of slicing is only throttled by libpcap‘s own read/write throughput as most of the remaining work is done in constant time. It took only 71 minutes (or 1.1 hours) to slice 1019 hosts out of a 180 GB pcap file on 2.5 GHz CPU. In simpler words, it’s lightning fast.

Right now the script does its job well enough. If someone needs to package it I’ll prefer removing the glib dependency in favor of perhaps glibc‘s own hash table implementation (search.h). In any case, I hope it proves helpful for other people playing with large pcap files.

Due to a number of commitments which I had pinned back during the summer for GSoC I was unable to attend much to the Internet over the past few weeks. Now that I’m back a summary of this year’s coding festival is in order:

The Program

This year I was working with Electronic Frontier Foundation/The Tor Project for improving the Anonymizing Relay Monitor (arm). The original proposal can be downloaded from this link are accessed via a browser at Google Docs. However, do note that not all of the goals from the proposal were met. Some were modified, some were removed altogether while some new ones were added — the details of which I’ll be explaining in the following sections.

Overall the program has been an extraordinarily enjoyable and learning experience for me. My involvement with Ubuntu last year had already taught me how invaluable it is to merge with your mentoring organizing’s developer community. This year most of my collaboration took place in #tor-dev on OFTC. Many times when I was stuck or heading towards an improper direction with my code the core Tor developers helped me and provided advice for design decisions as well as general guidance about the way things work in Tor. It wasn’t only a privilege to be helped by such rockstars, but was also vital as I can see in hindsight how disastrous it would have been if I had attempted to work through the program entirely on my own.

A huge thanks goes to my mentor Damian. Most of the credit for making this program an enjoyable and stimulating experience for me goes directly to him. He has one of the best combinations of code-people skills among people I’ve known. I would’ve loved meeting him and the Tor community in PETS ’11 but couldn’t travel due to some paperwork fiasco which was entirely a result of my slothful attitude towards anything involving government offices. Nevertheless, I do hope to meet the guys next year in PETS ’12.

The Code

In order to not sound repetitious, I’ll provide a quick summary of the milestones while linking to the posts which explain them in detail:

Menus for arm CLI

My first task was to add dropdown menus for the curses interface to arm. Even though the menus were replaced by Damian’s rewrite, they went a long way in helping me assimilate myself with the arm codebase:

(Click on the thumbnail for larger version.)

Graphs and logs for arm GUI

GTK+ was chosen as the toolkit for developing the arm GUI prototype. While GTK+ has its own disadvantages when compared to Qt (platform portability — or the lack thereof — being the foremost), it fared well in light-weight Unix environment such as Live CDs (e.g., Tails). Bandwidth graphs and logs for various arm events were added to the prototype:

(Click on the thumbnails for larger version.)

Connections and general stats for arm GUI

Moving on with the GUI, next up was to improve its conformity with the rest of the user’s desktop:

(Click on the thumbnail for larger version.)

And then re-use arm’s CLI connection resolvers in order to display stats about Tor circuits and control connections:

(Click on the thumbnail for larger version.)

A small addition was migration of the “sticky” panel from CLI which was moved under a “General” tab and provided miscellaneous info about Tor and arm:

(Click on the thumbnail for larger version.)

Configuration panel for GUI

Another important panel in the arm CLI was its configuration interface which provided a nice and documented approach to altering Tor’s settings. It was migrated to GUI with nice dialogs for validating user input:

(Click on the thumbnail for larger version.)

Along with the configuration panel a few patches to Tor and Vidalia were developed which would allow arm to be notified of changes made by an external program via a CONF_CHANGED event. The support for CONF_CHANGED still isn’t solid in all Tor controllers yet which I plan on addressing in coming months.

Exit node selector for arm CLI & GUI

A popular feature request among Tor users was to be able to select the country for their exit nodes. While I initially planned on providing them more fine-grained control over their circuits (such as path length), Tor developers advised against it and hence the selection was limited to the exit-node’s locale:

(Click on the thumbnails for larger version.)

The Nutshell

“It goes on forever — and — oh my God — it’s full of stars!”

It’s just that awesome, seriously. Stars from the FLOSS strata gather around and help inexperienced and aspiring developers all over the globe for two months in order to bring more code and — more importantly — more people to the open-source world. The experience with GSoC not only helps me in general open-source development, but also proves to be priceless at my workplace for my research in software defined networks. If you’re even remotely interested in open-source do keep an eye on the program’s website for future updates.

The final weeks of GSoC 2011 were spent by me working on exit node selection for Tor users. The GUI controller can now be used to define a list of countries, after which only those exit nodes shall be used which lie in one of the specified territories:

(Click on the thumbnail for larger version.)

For the CLI, Damian decided that the general use case for exit node selection is specification of a single country so pressing ‘E’ in the connections panel brings up a list from which one can be chosen:

(Click on the thumbnail for larger version.)

Please note that the exit node restriction works only for circuits built after the selection. Therefore it might be a good idea to send a NEWNYM to Tor after specifying the countries — which you’ll have to do manually for the time being until I add the feature to (g)arm controllers.

In my next post I’ll cover a summary of my involvement with GSoC this year, that is, after I finish with the regular chores of code submission etc.

While working for GSoC last year I kept track of Bazaar patches I sent in for Arsenal. This year I was using Git and as the need arose to generate a submission tarball for my commits I wrote this small utility script: git-generate-tarball.py.

To invoke the script, change into your Git repository and provide an author name and a file as an argument:

[krkhan@orthanc tor]$ /home/krkhan/Downloads/git-generate-tarball.py "Kamran Riaz Khan " /home/krkhan/Downloads/tor-gsoc-krkhan.tar.gz

generating tarball for commits between Mon May 23 00:00:00 2011 +0000 and Mon Aug 22 19:00:00 2011 +0000
generating patch for commit 5a801a8c8b71c9551a80913398135809cb10cecd
…
/home/krkhan/Downloads/tor-gsoc-krkhan.tar.gz created

The default starting and ending dates for the commits correspond to the schedule for GSoC 2011. You can specify a starting date for the commits you want to be packaged using the --since argument:

[krkhan@orthanc arm]$ /home/krkhan/Downloads/git-generate-tarball.py --since="August 1, 2011" "Kamran Riaz Khan " /home/krkhan/Downloads/arm-gsoc-krkhan.tar.bz2

generating tarball for commits between August 1, 2011 and Mon Aug 22 19:00:00 2011 +0000
generating patch for commit 546ca73259d7863e3efe5e11e09c023c2790a2f6
…
/home/krkhan/Downloads/arm-gsoc-krkhan.tar.bz2 created

Same goes for the --before argument:

[krkhan@orthanc tor]$ /home/krkhan/Downloads/git-generate-tarball.py --before="August 14, 2011" "Kamran Riaz Khan " /home/krkhan/Downloads/tor-gsoc-krkhan.tar.gz

generating tarball for commits between Mon May 23 00:00:00 2011 +0000 and August 14, 2011
generating patch for commit 5a801a8c8b71c9551a80913398135809cb10cecd
…
/home/krkhan/Downloads/tor-gsoc-krkhan.tar.gz created

Or a combination of both:

[krkhan@orthanc arm]$ /home/krkhan/Downloads/git-generate-tarball.py --since="August 1, 2011" --before="August 14, 2011" "Kamran Riaz Khan " /home/krkhan/Downloads/arm-gsoc-krkhan.tar.gz

generating tarball for commits between August 1, 2011 and August 14, 2011
generating patch for commit 546ca73259d7863e3efe5e11e09c023c2790a2f6
…
/home/krkhan/Downloads/arm-gsoc-krkhan.tar.gz created

If you want to leave the dates undefined you can leave the arguments empty. For example, the following command shall process all commits before June 1, 2011:

[krkhan@orthanc arm]$ /home/krkhan/Downloads/git-generate-tarball.py --since= --before="June 1, 2011" "Kamran Riaz Khan " /home/krkhan/Downloads/arm-gsoc-krkhan.tar.gz

generating tarball for commits before June 1, 2011
generating patch for commit 8b4dc162f75d5129e41f028c7253b7b265c8af76
…
/home/krkhan/Downloads/arm-gsoc-krkhan.tar.gz created

Hope this helps fellow GSoCers.