A Lazy Sequence

Dexie.js

Dexie is a very thin layer on top of the IndexedDB API. The primary advantage of Dexie is that, unlike the IndexedDB API, its not completely awful.

dexie.org

27 May 2016

Structured messaging followup

In my previous article I defined message types using TypeScript’s type alias facility. It turns out that interface extension is better here as it allows you to define recursive types where aliases do not. Simple change:

export type FooBar = IMessage<FooBarKeyT, {v: number}>;

becomes

export interface FooBar extends IMessage<FooBarKeyT, {v: number}> { }

The unioned types – such as `FooMessage` stay the same.

The Interfaces vs. Type Aliases section of the Advanced Types chapter in the TypeScript Handbook covers this briefly.

24 May 2016

Structured messages with TypeScript

I've been working on some architectural code for Manticore to move toward a more worker based structure. One aspect that I have been wanting to get cleaner is managing types for the messages that cross the worker boundaries.

One property of worker messages is that they use the ‘structured clone algorithm’ to share data between either side of the worker boundary. This allows data similar to JSON (although more types are supported) to be shared efficiently. Like JSON, you can pass objects but not classes. In TypeScript this means you necessarily lose type information.

Ideally I would like to be able to easily reconstitute type information for data received, and quickly make type-directed decisions. I also want to be able to work with messages generically when not at the boundaries of messaging, for instance, using a publish/subscribe hub. There are a few caveats here:

  • Without using some kind of schema or structural validation on top of what I present below, you need to trust both the sender and receiver to be generating structurally correct data. If you don't trust both sides (network requests, window message listeners, etc), you definitely want to include validation.
  • Without runtime types we have to perform all our dispatches using predicate functions.
  • Without generating code, theres little that can be done to avoid some of the boilerplate code needed.

You will want to be familiar with the advanced types section of the TypeScript handbook to grok the following; everything except intersection types and polymorphic this types are relevant.

All the code from this blog post is available in gist form.

The Message interface

To get started I am going to define an interface that all messages will use:

interface IMessage<TKey extends string, TPayload> {
  key: TKey;
  payload: TPayload;
}

A message has a key and some data. They key is used for routing and decision making with generic messages, and for determining the types with a typed message.

TKey extends string. The key is an instance of string so that we can use string literal types as part of the definition of typed message later.

Generic messages

This is pretty simple. GenericMessage is simply a type alias for messages where the key is any string, and the payload is any data:

type GenericMessage = IMessage<string, any>;

An example typed message

This is where the boilerplate starts. The definition of a typed message heavily leverages string literal types, union types, user-defined type guards, and type aliases. I place all the definitions for a particular group of message types within a single module:

module foo {
  type FooBarKeyT = "foo.bar";
  var FooBarKey: FooBarKeyT = "foo.bar";
  type FooQuuxKeyT = "foo.quux";
  var FooQuuxKey: FooQuuxKeyT = "foo.quux";
	
  export type FooBar = IMessage<FooBarKeyT, {v: number}>;
  export type FooQuux = IMessage<FooQuuxKeyT, {v: string}>;
  export type FooMessage = FooBar | FooQuux;


  // User-defined type guards:
  export function isFooMessage(m:GenericMessage): m is FooMessage {
    return m.key === FooBarKey 
        || m.key === FooQuuxKey;
  }
	
  export function isFooBarMessage(m:FooMessage): m is FooBar {
    return m.key === FooBarKey;
  }
	
  export function isFooQuuxMessage(m:FooMessage): m is FooQuux {
    return m.key === FooQuuxKey;
  }

	
  // Functions to create messages:
  export function fooBarMessage(v: number): FooMessage {
    return {key: FooBarKey, payload: {v: v}};
  }
	
  export function fooQuuxMessage(v: string): FooMessage {
    return {key: FooQuuxKey, payload: {v: v}};
  }
}

This is the guts of a typed message. This module, foo has two types of message – FooBar, and FooQuux – within the broader FooMessage type.

Defining the types

The first step is to define the message keys. This is done with the goofy pairs of lines at the top

type FooBarKeyT = "foo.bar";
const FooBarKey: FooBarKeyT = "foo.bar";
type FooQuuxKeyT = "foo.quux";
const FooQuuxKey: FooQuuxKeyT = "foo.quux";

The first pair of lines defines the type for the FooBar key by assigning a single string literal to a type alias. I've used a T suffix to separate it from the constant of the same name. The constant is then assigned the same string literal, and has the type explicitly provided. This ensures that the type and the constant cannot diverge, and that we don't need to repeat the literal any further. The process is repeated for FooQuux's key.

Note that there is no key defined for FooMessage. That type exists as a union of the above two keys.

export type FooBar = IMessage<FooBarKeyT, {v: number}>;
export type FooQuux = IMessage<FooQuuxKeyT, {v: string}>;
export type FooMessage = FooBar | FooQuux;

Next is defining the types for the messages themselves. Here I am just using type aliases of the IMessage interface to specify the key – using the key types just defined – and the payload. FooMessage is simply defined as the union of both IMessage types.

A very useful property of these message definitions is that the compiler will not let you specify the wrong keys, or, once the keys are specified, the wrong payload data. Try it out in a playground

Defining the guards

With the types defined, the next step is to define guard predicates for each type including the FooMessage union. This is total boilerplate but drastically improves the usability of the typed messages. You can see this in action in the consumer module of the gist.

export function isFooMessage(m:GenericMessage): m is FooMessage {
  return m.key === FooBarKey 
      || m.key === FooQuuxKey;
}
	
export function isFooBarMessage(m:FooMessage): m is FooBar {
  return m.key === FooBarKey;
}
	
export function isFooQuuxMessage(m:FooMessage): m is FooQuux {
  return m.key === FooQuuxKey;
}

Note that the FooMessage guard takes a GenericMessage argument. This means you can union FooMessages with all the other messages you expect to receive and easily dispatch based on type.

Message construction functions

These two functions – fooBarMessage, and fooQuuxMessage – are self explanatory. They return plain objects rather than using classes because when they are posted across a worker boundary this is what the other side will receive anyway.


That’s everything. I am still exploring this idea. So far it appears to have merit in spite of the boilerplate required. If you make use of this idiom in your projects I would be interested to hear how it goes.

In a future post I may write up some notes on how I am using workers, and a lightweight worker class I have written to structure my program.

19 May 2016

Black boxes for guitar

If you've ever participated in guitarist culture, especially online, you'll know that we love knowing what ‘significant’ components make up the internals of their gear: Tubes, JRC4558D and LM308s op amps, germanium transistors and diodes, spring reverb tanks… the list of mojo parts seems like it may be endless.

One aspect of the guitarist culture that reinforces this is the received wisdom we all pick up over time. Understanding the various and mysterious boxes we plug our instruments into falls outside our expertise. To make up for this a cargo cult mentality about the easily identifiable aspects of a device has sprung up.

By being so concerned with the inventory of what’s inside the box, rather than just the sounds, has, I think, blinded guitarists to potential developments and improvements. There’s no more obvious place than amplifiers and the vacuum tube.

Certainly, many early solid state and digital amplifiers did tend to sound pretty terrible, but not due to inherent failings of solid state technology specifically, simply its application1. Yes: replacing a tube with a transistor and expecting to get the same results is often an exercise in futility. Instead of substituting device for device, newer designs look to replace circuits with circuits. This is not some radical new idea either. Science, engineering, and computing have used the notion of a black box in designing and analysing systems for a very long time.

In saying this, I don’t want to imply that it‘s not useful to know about the circuits and internals. The opposite: I’m more interested in how these tools work now than I ever have been2. I simply want to be open to great guitar gear based on its sound, not what is in it, and that’s why I’m no longer a snob about tube amps.

Footnotes

  1. It is worth keeping in mind too that most people encounter solid state and digital amps as cheap starter gear. It’s obviously not a fair comparison. For one thing, the speakers used in starter amps are generally complete rubbish.
  2. It’s also nice to be able to avoid yet another Tube Screamer with minor tweak.