Monday, September 14, 2009

Patterns in Network Architecture

I recently finished reading Patterns in Network Architecture by John Day. It's an attempt to rethink network architectures and polish up "the unfinished demo" that is the internet.

Now, I'm not a network guy, so I can't evaluate the quality of his proposed solutions in any detail, but I liked his thought process and found it a useful read for anyone interested in a good example of thinking through a hard problem and coming up with a disciplined "minimal covering" solution.

Day focuses upon discovering the appropriate layers and layer structures necessary for communication. He works up from interprocess communication on a single machine to processes communicating across multiple machines.

The implications of this analysis are interesting in and of themselves and closely resemble structures seen in other systems. His metaphors are primarily in terms of name lookup and binding, using compilers and operating systems as examples (I have to admit that this only feels partially correct to me: I think the full problem is more akin to providing the data/instructions to a processor and therefore needs to include the mapping from a "memory location" to an actual address accessible by the chip's execution unit e.g., it should take into account caching, TLBs et al.).

The first and foremost conclusion in Day's opinion is that there is one layer that provides interprocessor communication and it replicates. That is, the structure of each network layer is the same, but the policies and optimizations differ depending upon the particulars of what the layer is connected to. Every layer has three parts, data transfer, IPC (Interprocess Communication) control and IPC management -- where control is short cycle management.

In his words

"Layers have two major properties that are of interest to us: abstraction and scaling (i.e., divide and conquer). Layers hide the operation of the internal mechanisms from the users of the mechanisms and segregate and aggregate traffic. But most important, they provide an abstraction of the layers below"

When moving from a shared to a distributed environment the core new functionality is an Error- and flow-control protocol (EFCP). This protocol replaces the shared memory mechanisms of the single system to ensure reliability and to provide flow control in the environment of communication between two systems. An EFCP PM is a task of the IPC process. Although in theory, such a process could be included even when communication is on a shared processor, in practice, this communication is so reliable as to make it redundant.

I think that the core insight/technique was to frame communication from the network perspective as being from application to application and not as interacting with the network e.g., in the figure below (6-15 from the book) communication is conceptualized as being across, that is between applications at the same layer, rather than down through the network and back up to the other application.

The application concerns itself with developing a shared state with its partner application. The N-1 layer provides an an abstract aggregated API to support the application's view of the communication and performs whatever aggregation and abstraction necessary to develop a shared state with the N-1 application on the other side. It then hands off details to the N-2 layer which gets it to the N-2 layer on the other side, etc.


Yes, it is just encapsulation all over again, but as we all know, finding the right thing to encapsulate and doing it in a practical way takes a lot of work.

I'm eliding a number of the other key findings of the book such as
  • The observation than an address only needs to be unique within a (distributed) application layer
  • A connection is made only after authentication has been obtained and the connection authorized etc.

All are developed in a thoughtful way showing deep insight into the problem.

Although not the easiest read for someone without a strong networking background, it is an interesting and useful exercise to watch someone so well versed go through the process so thoughtfully.

No comments: