2013-05-09: HTTP Mailbox - Asynchronous RESTful Communication

By -
We often encounter web services that take a very long time to respond to our HTTP requests. In the case of an eventual network failure, we are forced to issue the same HTTP request again. We frequently consume web services that do not support REST. If they did, we could utilize the full range of HTTP methods while retaining the functionality of our application, even when the external API we utilize in our application changes. We sometime wish to set up a web service that takes job requests, processes long running job queues and notifies the clients individually or in groups. HTTP does not allow multicast or broadcast messaging. HTTP also requires the client to stay connected to the server while the request is being processed.

Introducing HTTP Mailbox - An Asynchronous RESTful HTTP Communication System. In a nutshell, HTTP Mailbox is a mailbox for HTTP messages. Using its RESTful API, anyone can send an HTTP message (request or response) to anyone else independent of the availability, or even the identity of recipient(s). The HTTP Mailbox stores these messages and delivers them on demand. Each HTTP message is encapsulated in the body of another HTTP message and sent to the HTTP Mailbox using a POST method. Similarly, the HTTP Mailbox encapsulates the HTTP message in the body of its response when a GET request is made to retrieve the messages.

Tunneling HTTP traffic over HTTP was also explored in the Relay HTTP. But the Relay HTTP relays the live HTTP traffic back and forth and does not store HTTP messages. It works like a proxy server to only overcome JavaScript's cross-origin restriction in Ajax requests. The Relay HTTP still requires the client and server along with the relay server to meet in time.

Store and forward nature of the HTTP Mailbox is inspired by Linda. We have taken the simplicity of Linda model and implemented it using HTTP on the scale of the Web. This approach has enabled asynchronous, indirect, time-uncoupled, space-uncoupled, individual, and group communication over HTTP. Time-uncoupling refers to no need of sender and recipient(s) meeting in time to communicate while space-uncoupling refers to no need of sender and recipient(s) knowing each other's identity to communicate. The HTTP Mailbox also enabled utilization of the full range of HTTP methods otherwise unavailable to standard clients and servers.


The above figure shows the lifecycle of a typical HTTP message using the HTTP Mailbox in four steps. We will walk through this example to explain how it works. Assume that the client wants to send the following HTTP message to the server at example.com/tasks/1.

> PATCH /tasks/1 HTTP/1.1
> Host: example.com
> Content-Type: text/task-patch
> Content-Length: 11
> 
> Status=Done
Step 1, assuming that the HTTP Mailbox server is running on example.net, therefore the message will be encapsulated in a POST message like this:

> POST /hm/http://example.com/tasks HTTP/1.1
> Host: example.net
> HM-Sender: http://example.org/alice
> Content-Type: message/http; msgtype: request
> Content-Length: 108
> 
> PATCH /tasks/1 HTTP/1.1
> Host: example.com
> Content-Type: text/task-patch
> Content-Length: 11
> 
> Status=Done
Step 2, the HTTP Mailbox will store the message and return the URI of newly created message in the response as follows:

< HTTP/1.1 201 Created
< Location: http://example.net/hm/id/5ecb44e0
< Date: Thu, 20 Dec 2012 02:22:56 GMT
 
Step 3, example.com makes an HTTP GET request to the HTTP Mailbox server to retrieve its messages. To retrieve the most recent message sent to "http://example.com/tasks" a request will look like this:

> GET /hm/http://example.com/tasks HTTP/1.1
> Host: example.net
Step 4, the response from the HTTP Mailbox will contain the most recent message sent to "http://example.com/tasks".  The response will also include a "Link" header that will give the URLs to navigate through the chain of messages for that recipient.

< HTTP/1.1 200 OK
< Date: Thu, 20 Dec 2012 02:10:22 GMT
< Link: <http://example.net/hm/id/aebed6e9>; rel="first",
<  <http://example.net/hm/id/5ecb44e0>; rel="last self",
<  <http://example.net/hm/id/85addc19>; rel="previous",
<  <http://example.net/hm/http://example.com/tasks>; rel="current"
< Via: Sent by 127.0.0.1
<  on behalf of http://example.org/alice
<  delivered by http://example.net/
< Content-Type: message/http; msgtype: request
< Content-Length: 108
< 
< PATCH /tasks/1 HTTP/1.1
< Host: example.com
< Content-Type: text/task-patch
< Content-Length: 11
< 
< Status=Done
A tech report is published on arXiv, describing the HTTP Mailbox in details. A reference implementation of the HTTP Mailbox can be found on GitHub.

We have already used the HTTP Mailbox successfully in the following applications.
  • Preserve Me! - a distributed web object preservation system that establishes social network among web objects and uses the HTTP Mailbox for its communication needs.
  • Preserve Me! Viz - a dynamic and interactive network graph visualization tool to give insight of the Preserve Me! graph and communication.


Where else can we use the HTTP Mailbox?
  • Warrick - a tool to restore lost websites. It can use HTTP Mailbox to accept service requests and status notification.
  • Carbon Dating the Web - a tool to find out the age of a resource at a given URL. This process usually takes few minutes to complete each request in the queue. It can utilize the HTTP Mailbox to accept service requests and send the response when ready.
  • Device notifications - related to software updates, general application messaging.
  • Any application that needs asynchronous RESTful HTTP messaging.

Resources

--
Sawood Alam

Comments

Post a Comment