Inspirated

 
 

May 28, 2010

Summer of Code Progress: REST collections

Filed under: Blog — krkhan @ 6:44 pm

Starting from this post, I will be documenting my SoC progress according to the Ubuntu guidelines. This will enable me to log my activities in an organized manner. The inaugural report:

Related Links

Summer of Code Archive Inspirated Code
Report Guidelines Ubuntu Wiki
Original Proposal Ubuntu Wiki

Time Spent

8 hours.

Highlights

This is my first experience of developing on a Zope project. The fact that the project is already deployed in a large-scale production environment is daunting as well as exciting at the same time. For Launchpad work, I learned only the bare essentials of Zope, i.e., things that are directly related with my project such as interfaces and adapters. After SoC however, I plan on exploring it in more detail.

Concerns

Testing Launchpad modifications is a beast of a task. After making each set of changes to the code, I have to stop the server and relaunch it again which takes at least 30 seconds. It becomes especially frustrating when I’m in the process of making tiny changes in order to explore the API.

I’ve tried to work around this issue by using pdb.set_trace(). Still, it’s no substitute for being able to edit whole files in one go.

Waiting Items

None.

Stalled Tasks

None.

Accomplishments

  • Test locally running Launchpad with a RESTClient.
  • Go through the code-base and figure out the portions which deal with API services.
  • Export a bug collection for every project: 
    1
2
3
4
5
6
7
8
9
10
11
12
13
14

    <rest-client version="2.3"><request>
<http-version>1.1</http-version>
<URL>https://api.launchpad.dev/beta/alsa-utils/bugs</URL>
<method>GET</method>
<ssl-truststore>/home/inspirated/.keystore</ssl-truststore>
<ssl-truststore-password>rO0ABXQAB2tyYWNrM2Q=</ssl-truststore-password>
<ssl-hostname-verifier>ALLOW_ALL</ssl-hostname-verifier>
<headers>
<header key="Authorization" value="OAuth realm="OAuth", oauth_nonce="FFFF", oauth_timestamp="1275040360", oauth_consumer_key="just%20testing", oauth_signature_method="PLAINTEXT", oauth_version="1.0", oauth_token="0NWl33zGj3QDtxFWdHCt", oauth_signature="%261XP9XCptHQ3S8g49RjFTr3xFlrK0H73Z5QjbGbxs1pMTc5mzptq0Z5Qs93gjcXDQtqlgFSVVQBPLdK47""/>
<header key="Accept" value="application/json"/>
<header key="Host" value="api.launchpad.dev"/>
</headers>
</request>
</rest-client>
    1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104

    <rest-client version="2.3">
<response>
<execution-time>1833</execution-time>
<status code="200">HTTP/1.1 200 Ok</status>
<headers>
<header key="Date" value="Fri, 28 May 2010 12:50:18 GMT"/>
<header key="Server" value="zope.server.http (DebugLayerHTTP)"/>
<header key="X-Powered-By" value="Zope (www.zope.org), Python (www.python.org)"/>
<header key="Content-Type" value="application/json"/>
<header key="Content-Length" value="10736"/>
<header key="Vary" value="Accept,Accept-Encoding"/>
<header key="Via" value="1.1 launchpad.dev"/>
<header key="Keep-Alive" value="timeout=15, max=100"/>
<header key="Connection" value="Keep-Alive"/>
</headers>
<body>
{
   total_size           : 15,
   start                : 0,
   resource_type_link   : 'https://api.launchpad.dev/beta/#bug-page-resource',
   next_collection_link : 'https://api.launchpad.dev/beta/alsa-utils/bugs?ws.start=5&amp;ws.size=5',
   entries              : [
      {
         users_unaffected_collection_link          : 'https://api.launchpad.dev/beta/bugs/15/users_unaffected',
         latest_patch_uploaded                     : null,
         users_affected_count_with_dupes           : 0,
         security_related                          : false,
         private                                   : false,
         bug_watches_collection_link               : 'https://api.launchpad.dev/beta/bugs/15/bug_watches',
         date_made_private                         : null,
         linked_branches_collection_link           : 'https://api.launchpad.dev/beta/bugs/15/linked_branches',
         subscriptions_collection_link             : 'https://api.launchpad.dev/beta/bugs/15/subscriptions',
         number_of_duplicates                      : 0,
         id                                        : 15,
         users_unaffected_count                    : 0,
         title                                     : 'Nonsensical bugs are useless',
         name                                      : null,
         http_etag                                 : '"23e175849b60b27f5553ddb56995e044c71d745a-ce4f09c837a78ed5fdf23f759020058f3423a804"',
         messages_collection_link                  : 'https://api.launchpad.dev/beta/bugs/15/messages',
         self_link                                 : 'https://api.launchpad.dev/beta/bugs/15',
         who_made_private_link                     : null,
         attachments_collection_link               : 'https://api.launchpad.dev/beta/bugs/15/attachments',
         resource_type_link                        : 'https://api.launchpad.dev/beta/#bug',
         date_last_updated                         : '2007-12-18T16:31:34.972893+00:00',
         description                               : 'Like this one, natch.',
         duplicates_collection_link                : 'https://api.launchpad.dev/beta/bugs/15/duplicates',
         tags                                      : [],
         message_count                             : 7,
         heat                                      : 0,
         bug_tasks_collection_link                 : 'https://api.launchpad.dev/beta/bugs/15/bug_tasks',
         cves_collection_link                      : 'https://api.launchpad.dev/beta/bugs/15/cves',
         users_affected_with_dupes_collection_link : 'https://api.launchpad.dev/beta/bugs/15/users_affected_with_dupes',
         duplicate_of_link                         : null,
         users_affected_count                      : 0,
         owner_link                                : 'https://api.launchpad.dev/beta/~name16',
         date_created                              : '2007-12-18T16:30:19.103679+00:00',
         can_expire                                : false,
         date_last_message                         : '2007-12-18T16:31:34.790641+00:00',
         users_affected_collection_link            : 'https://api.launchpad.dev/beta/bugs/15/users_affected'
      },
      <!-- entries omitted -->
      {
         users_unaffected_collection_link          : 'https://api.launchpad.dev/beta/bugs/11/users_unaffected',
         latest_patch_uploaded                     : null,
         users_affected_count_with_dupes           : 0,
         security_related                          : false,
         private                                   : false,
         bug_watches_collection_link               : 'https://api.launchpad.dev/beta/bugs/11/bug_watches',
         date_made_private                         : null,
         linked_branches_collection_link           : 'https://api.launchpad.dev/beta/bugs/11/linked_branches',
         subscriptions_collection_link             : 'https://api.launchpad.dev/beta/bugs/11/subscriptions',
         number_of_duplicates                      : 0,
         id                                        : 11,
         users_unaffected_count                    : 0,
         title                                     : 'Make Jokosher use autoaudiosink',
         name                                      : null,
         http_etag                                 : '"8675b8d2649cf0c08d58681036ef1e8bb6a20e78-d7d969f7ed8c2f2def4d9e429f23ac41f7fec4dc"',
         messages_collection_link                  : 'https://api.launchpad.dev/beta/bugs/11/messages',
         self_link                                 : 'https://api.launchpad.dev/beta/bugs/11',
         who_made_private_link                     : null,
         attachments_collection_link               : 'https://api.launchpad.dev/beta/bugs/11/attachments',
         resource_type_link                        : 'https://api.launchpad.dev/beta/#bug',
         date_last_updated                         : '2007-03-15T20:37:51.603369+00:00',
         description                               : 'I\'ve had problems when switching from Jokosher to Totem to play an Ogg.\n\nTotem appears to be playing normally but does not produce any sound.\nIf I close Jokosher then you can hear totem.\n\nI\'ve also had a problem when trying to playback sound within jokosher\nafter switch from another app (I think it was xmms). I get a dialog\nwith the following text:\n\n     Argh! Something went wrong and a serious error occurred:\n\n     Resource busy or not available.\n\n     gstalsasink.c(636): gst_alsasink_open (): /timeline/playbackbin/\n     alsasink0:\n     Device \'hw:0\' is busy\n\nAfter closing and reopening Jokosher, i got the same error again.\nAfter logging out of gnome and logging in again - I still got the same\nerror message.\n\nI had to restart!\n\nIs there some way to reset alsa or the device driver - with out having\nto restart?\n\nRunning on Ubuntu - with Jokosher 0.2 runscript.',
         duplicates_collection_link                : 'https://api.launchpad.dev/beta/bugs/11/duplicates',
         tags                                      : [],
         message_count                             : 7,
         heat                                      : 0,
         bug_tasks_collection_link                 : 'https://api.launchpad.dev/beta/bugs/11/bug_tasks',
         cves_collection_link                      : 'https://api.launchpad.dev/beta/bugs/11/cves',
         users_affected_with_dupes_collection_link : 'https://api.launchpad.dev/beta/bugs/11/users_affected_with_dupes',
         duplicate_of_link                         : null,
         users_affected_count                      : 0,
         owner_link                                : 'https://api.launchpad.dev/beta/~name16',
         date_created                              : '2007-03-15T20:33:56.678930+00:00',
         can_expire                                : false,
         date_last_message                         : null,
         users_affected_collection_link            : 'https://api.launchpad.dev/beta/bugs/11/users_affected'
      }
   ]
}
</body>
</response>
</rest-client>

Minor Tasks

Having gotten stuck on the task of running RESTClient against my Launchpad instance, I ran into a few issues such as RESTClient’s refusal to accept the local SSL certificate and confusion regarding domain names. The former issue was solved by using keytool to create a trusted keystore while the latter was addressed by using correct domain names as pointed out by William Grant.

Actions for the Following Report

While I have managed to export a web service property for Launchpad projects, I haven’t yet figured out a way to group bugs by projects. This will require a clear understanding of the schemas involved as well as some skills with SQL. My next task therefore will be to export only the bugs that are related to a particular project.

Tags: , , , , , , , ,

May 22, 2010

Summer of Code Progress: Debugging Launchpad’s RESTful API

Filed under: Blog — krkhan @ 8:03 pm

Related Links

Summer of Code Archive Inspirated Code
Original Proposal Ubuntu Wiki

Report

RESTful web services are very fun once they get properly integrated in end-user applications. Testing those services is a whole different thing and I wanted to dive deep into the bare essentials of REST communication performed by Launchpad for its API. The most straightforward way was to write a Python script using urllib2 and/or httplib for the HTTP request/response mantra. Obviously I found it to be a cumbersome solution and then found this little beauty:

RESTClient Screenshot
(Click on the thumbnail for larger version.)

RESTClient does one thing and does it sweetly: it gives you a nice GUI for toying with REST requests. Getting it to work with Launchpad was not as straightforward though as all requests have to be signed to be of any real value. I also had to spent a few hours trying to understand why api.edge.launchpad.net was not recognizing my access tokens until by sheer stroke of luck I used api.staging.launchpad.net and it started working perfectly.

1
2
3
4
5
6
7
8
9
10
11
12

<rest-client version="2.3">
<request>
<http-version>1.1</http-version>
<URL>https://api.staging.launchpad.net/beta/bugs/1</URL>
<method>GET</method>
<headers>
<header key="Authorization" value="OAuth realm="OAuth", oauth_nonce="77848601", oauth_timestamp="1274537900", oauth_consumer_key="just%20testing", oauth_signature_method="PLAINTEXT", oauth_version="1.0", oauth_token="6dwHFn3CPzxjNrrdp3r9", oauth_signature="%26xxxxxx""/>
<header key="Accept" value="application/json"/>
<header key="Host" value="api.staging.launchpad.net"/>
</headers>
</request>
</rest-client>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

<rest-client version="2.3">
<response>
<execution-time>897</execution-time>
<status code="200">HTTP/1.1 200 Ok</status>
<headers>
<header key="Date" value="Sat, 22 May 2010 14:26:32 GMT"/>
<header key="Server" value="zope.server.http (HTTP)"/>
<header key="X-Powered-By" value="Zope (www.zope.org), Python (www.python.org)"/>
<header key="Content-Type" value="application/json"/>
<header key="Content-Length" value="2888"/>
<header key="Etag" value=""d9b357c932ada0a71e96401a8c87368d4704a500-dec69b947b887c4addba8eb4aae5ca64c94f616c""/>
<header key="Vary" value="Accept,Accept-Encoding"/>
<header key="Via" value="1.1 wildcard.staging.launchpad.net"/>
<header key="Keep-Alive" value="timeout=15, max=100"/>
<header key="Connection" value="Keep-Alive"/>
</headers>
<body>
{
   users_unaffected_collection_link          : 'https://api.staging.launchpad.net/beta/bugs/1/users_unaffected',
   latest_patch_uploaded                     : null,
   users_affected_count_with_dupes           : 378,
   security_related                          : false,
   private                                   : false,
   bug_watches_collection_link               : 'https://api.staging.launchpad.net/beta/bugs/1/bug_watches',
   date_made_private                         : null,
   linked_branches_collection_link           : 'https://api.staging.launchpad.net/beta/bugs/1/linked_branches',
   subscriptions_collection_link             : 'https://api.staging.launchpad.net/beta/bugs/1/subscriptions',
   number_of_duplicates                      : 0,
   id                                        : 1,
   users_unaffected_count                    : 6,
   title                                     : 'Microsoft has a majority market share',
   name                                      : 'liberation',
   http_etag                                 : '"d9b357c932ada0a71e96401a8c87368d4704a500-dec69b947b887c4addba8eb4aae5ca64c94f616c"',
   messages_collection_link                  : 'https://api.staging.launchpad.net/beta/bugs/1/messages',
   self_link                                 : 'https://api.staging.launchpad.net/beta/bugs/1',
   who_made_private_link                     : null,
   attachments_collection_link               : 'https://api.staging.launchpad.net/beta/bugs/1/attachments',
   resource_type_link                        : 'https://api.staging.launchpad.net/beta/#bug',
   date_last_updated                         : '2010-05-13T23:24:26.362094+00:00',
   description                               : 'Microsoft has a majority market share in the new desktop PC marketplace.\nThis is a bug, which Ubuntu is designed to fix.\n\nNon-free software is holding back innovation in the IT industry, restricting access to IT to a small part of the world\'s population and limiting the ability of software developers to reach their full potential, globally. This bug is widely evident in the PC industry.\n\nSteps to repeat:\n\n1. Visit a local PC store.\n\nWhat happens:\n2. Observe that a majority of PCs for sale have non-free software pre-installed.\n3. Observe very few PCs with Ubuntu and free software pre-installed.\n\nWhat should happen:\n1. A majority of the PCs for sale should include only free software like Ubuntu.\n2. Ubuntu should be marketed in a way such that its amazing features and benefits would be apparent and known by all.\n3. The system shall become more and more user friendly as time passes.\n\n',
   duplicates_collection_link                : 'https://api.staging.launchpad.net/beta/bugs/1/duplicates',
   tags                                      : [
      'iso-testing',
      'ubuntu'
   ],
   message_count                             : 1198,
   heat                                      : 2138,
   bug_tasks_collection_link                 : 'https://api.staging.launchpad.net/beta/bugs/1/bug_tasks',
   cves_collection_link                      : 'https://api.staging.launchpad.net/beta/bugs/1/cves',
   users_affected_with_dupes_collection_link : 'https://api.staging.launchpad.net/beta/bugs/1/users_affected_with_dupes',
   duplicate_of_link                         : null,
   users_affected_count                      : 378,
   owner_link                                : 'https://api.staging.launchpad.net/beta/~sabdfl',
   date_created                              : '2004-08-20T00:00:00+00:00',
   can_expire                                : false,
   date_last_message                         : '2010-05-13T23:24:21.253673+00:00',
   users_affected_collection_link            : 'https://api.staging.launchpad.net/beta/bugs/1/users_affected'
}
</body>
</response>
</rest-client>

With a local Launchpad ready to do my bidding and a debugging process in place for elaborating its API, I’m looking forward to the coding phase. Which, coincidentally, begins on the same day I give my final exam for the current semester.

“Rest REST: the sweet sauce of labor.” — Plutarch

Tags: , , , , , , , , ,
« Previous Page
 
Creative Commons License

This work is licensed under a Creative Commons Attribution-Share Alike 3.0 License