A Lazy Sequence

Structured messages with TypeScript

I've been working on some architectural code for Manticore to move toward a more worker based structure. One aspect that I have been wanting to get cleaner is managing types for the messages that cross the worker boundaries.

One property of worker messages is that they use the ‘structured clone algorithm’ to share data between either side of the worker boundary. This allows data similar to JSON (although more types are supported) to be shared efficiently. Like JSON, you can pass objects but not classes. In TypeScript this means you necessarily lose type information.

Ideally I would like to be able to easily reconstitute type information for data received, and quickly make type-directed decisions. I also want to be able to work with messages generically when not at the boundaries of messaging, for instance, using a publish/subscribe hub. There are a few caveats here:

You will want to be familiar with the advanced types section of the TypeScript handbook to grok the following; everything except intersection types and polymorphic this types are relevant.

All the code from this blog post is available in gist form.

The Message interface

To get started I am going to define an interface that all messages will use: 

interface IMessage<TKey extends string, TPayload> {
  key: TKey;
  payload: TPayload;
}

A message has a key and some data. They key is used for routing and decision making with generic messages, and for determining the types with a typed message.

TKey extends string. The key is an instance of string so that we can use string literal types as part of the definition of typed message later.

Generic messages

This is pretty simple. GenericMessage is simply a type alias for messages where the key is any string, and the payload is any data: 

type GenericMessage = IMessage<string, any>;

An example typed message

This is where the boilerplate starts. The definition of a typed message heavily leverages string literal types, union types, user-defined type guards, and type aliases. I place all the definitions for a particular group of message types within a single module: 

module foo {
  type FooBarKeyT = "foo.bar";
  var FooBarKey: FooBarKeyT = "foo.bar";
  type FooQuuxKeyT = "foo.quux";
  var FooQuuxKey: FooQuuxKeyT = "foo.quux";
	
  export type FooBar = IMessage<FooBarKeyT, {v: number}>;
  export type FooQuux = IMessage<FooQuuxKeyT, {v: string}>;
  export type FooMessage = FooBar | FooQuux;


  // User-defined type guards:
  export function isFooMessage(m:GenericMessage): m is FooMessage {
    return m.key === FooBarKey 
        || m.key === FooQuuxKey;
  }
	
  export function isFooBarMessage(m:FooMessage): m is FooBar {
    return m.key === FooBarKey;
  }
	
  export function isFooQuuxMessage(m:FooMessage): m is FooQuux {
    return m.key === FooQuuxKey;
  }

	
  // Functions to create messages:
  export function fooBarMessage(v: number): FooMessage {
    return {key: FooBarKey, payload: {v: v}};
  }
	
  export function fooQuuxMessage(v: string): FooMessage {
    return {key: FooQuuxKey, payload: {v: v}};
  }
}

This is the guts of a typed message. This module, foo has two types of message – FooBar, and FooQuux – within the broader FooMessage type.

Defining the types

The first step is to define the message keys. This is done with the goofy pairs of lines at the top 

type FooBarKeyT = "foo.bar";
const FooBarKey: FooBarKeyT = "foo.bar";
type FooQuuxKeyT = "foo.quux";
const FooQuuxKey: FooQuuxKeyT = "foo.quux";

The first pair of lines defines the type for the FooBar key by assigning a single string literal to a type alias. I've used a T suffix to separate it from the constant of the same name. The constant is then assigned the same string literal, and has the type explicitly provided. This ensures that the type and the constant cannot diverge, and that we don't need to repeat the literal any further. The process is repeated for FooQuux's key.

Note that there is no key defined for FooMessage. That type exists as a union of the above two keys. 

export type FooBar = IMessage<FooBarKeyT, {v: number}>;
export type FooQuux = IMessage<FooQuuxKeyT, {v: string}>;
export type FooMessage = FooBar | FooQuux;

Next is defining the types for the messages themselves. Here I am just using type aliases of the IMessage interface to specify the key – using the key types just defined – and the payload. FooMessage is simply defined as the union of both IMessage types.

A very useful property of these message definitions is that the compiler will not let you specify the wrong keys, or, once the keys are specified, the wrong payload data. Try it out in a playground

Defining the guards

With the types defined, the next step is to define guard predicates for each type including the FooMessage union. This is total boilerplate but drastically improves the usability of the typed messages. You can see this in action in the consumer module of the gist. 

export function isFooMessage(m:GenericMessage): m is FooMessage {
  return m.key === FooBarKey 
      || m.key === FooQuuxKey;
}
	
export function isFooBarMessage(m:FooMessage): m is FooBar {
  return m.key === FooBarKey;
}
	
export function isFooQuuxMessage(m:FooMessage): m is FooQuux {
  return m.key === FooQuuxKey;
}

Note that the FooMessage guard takes a GenericMessage argument. This means you can union FooMessages with all the other messages you expect to receive and easily dispatch based on type.

Message construction functions

These two functions – fooBarMessage, and fooQuuxMessage – are self explanatory. They return plain objects rather than using classes because when they are posted across a worker boundary this is what the other side will receive anyway.

That’s everything. I am still exploring this idea. So far it appears to have merit in spite of the boilerplate required. If you make use of this idiom in your projects I would be interested to hear how it goes.

In a future post I may write up some notes on how I am using workers, and a lightweight worker class I have written to structure my program.

20 May 2016