Retrieving and working with the tokens

Get a SDTFClient

The SDTFClient is a class providing numerous methods to work with the design data stored using the Specify Design Token Format (SDTF).

It's the way of interacting with your tokens and generating content.

// Fetch a Repository
const SPECIFY_REPOSITORY_NAME = "MY-REPOSITORY-NAME";
const sdtfClient = await specifyClient.getSDTFClientByRepositoryName(
  SPECIFY_REPOSITORY_NAME,
);

console.log(`Fetched repository: ${sdtfClient.repository.name}`);

Get the SDTF JSON token tree

You would want to grab the SDTF JSON token tree of a repository anytime you want to: send the token tree over the network or debug an intermediate manipulation.

const jsonTokenTree = sdtfClient.getJSONTokenTree();

console.log(jsonTokenTree) // the object literal representation of the Specify Design Token data

Now that we know how to retrieve our SDTF, let's see how we can manipulate and retrieve our tokens.

Retrieving the tokens

There's multiple way to retrieve a token. If you want to work with a specific one you'll probably want to only get one, if you're developing a generic solution, you'll probably prefer iterating over many tokens. Let's have a look on how to do this.

Retrieving a single token

To retrieve a single token, you can just pick it by its path:

const tokenState = sdtfClient.getTokenState(['path', 'to', 'token'])

Retrieving all the tokens

Tokens

const tokens = sdtfClient.getAllTokenStates()

Groups

const groups = sdtfClient.getAllGroupStates()

Collections

const collections = sdtfClient.getAllCollectionStates()

Retrieving specific tokens

If you need specifically some tokens, for example, based on the name, type, path, etc... You can perform a query and map over the results:

const results = sdtfClient
  .mapQueryResults(
    { where: { collection: '^My Collection$', select: true, children: { tokens: true } } },
    (treeNodeState, engine, queryResult) => {
      if (treeNodeState.isCollection) {
        return treeNodeState.name
      }
      
      if (treeNodeState.isToken) {
        return treeNodeState.stringPath
      }
      
      return undefined
    },
  );

Or, you can only iterate over the results of your query if you need to perform mutations for example:

sdtfClient
  .forEachQueryResult(
    { where: { token: '.*', select: true } },
    (treeNodeState, engine, queryResult) => {
      if (treeNodeState.isToken) {
        engine.mutation.renameToken({
          atPath: treeNodeState.path,
          name: treeNodeState.name.toLowerCase(),
        });
      }
    },
  );

Looking for details about the query? 👉 heads up to the SDTF Query Language concept.

Looking for more methods to retrieve tokens? 👉 heads up to the SDTFEngine reference.

The token architecture

The TokenState instance will be your main access to your tokens. Through this, you can retrieve your token data or perform updates. But first, let’s put some context and try to figure out what this TokenState is trying to solve.

If we take a look at the token representation in the SDTF, it’s basically a JSON value where a lot of keys can be an alias rather than a value. Let’s have a look at all the possible cases with an example. Note that the following examples are 4 different ways of achieving the same result from a value point of view.

A basic token

{
  mySize: {
    $type: 'dimension',
    $value: {
      small: { unit: 'px', value: 12 },
      large: { unit: 'px', value: 24 },
    }
  }
}

An alias at the value level

{
  sizeValue: {
    $type: 'number',
    $value: {
      default: 12
    }
  },
  mySize: {
    $type: 'dimension',
    $value: {
      small: {
	unit: 'px',
	value: { $alias: 'sizeValue', $mode: 'default' }
      },
      large: {
	unit: 'px',
	value: 24
      },
    }
  }
}

An alias at the mode level

{
  anotherSize: {
    $type: 'dimension',
    $value: {
      customMode: {
	unit: 'px',
	value: 12
      }
    }
  },
  mySize: {
    $type: 'dimension',
    $value: {
      small: { $alias: 'anotherSize', $mode: 'customMode' },
      large: {
	unit: 'px',
	value: 24
      },
    }
  }
}

An alias at the top level

{
  anotherSize: {
    $type: 'dimension',
    $value: {
      small: {
	unit: 'px',
	value: 12
       },
       large: {
         unit: 'px',
	 value: 24
       },
    }
  },
  mySize: {
    $type: 'dimension',
    $value: { $alias: 'anotherSize' },
  }
}

As you can see, there’s a lot of way to express a token, and dealing with all the possible cases is quite difficult. That’s why one of the goal of the TokenState is to help you to deal with everything.

How to differentiate the tokens?

Before even thinking about retrieving the value of a token, we need to know what type of token we’re working with.

It’s actually quite easy to know as there’s a type property in the TokenState :

console.log('Token type:', tokenState.type) // Token type: <type>

But even if we know the type, a problem will remain: Typescript doesn’t. TokenState is quite a generic class as the default type is something that looks like this: TokenState<'dimension' | 'font' | 'breakpoint' | ...> .

So the first mission is to be able to narrow down the type to be able to work with it. To do so, there’s a pretty convenient function:

tokenState.matchByType(
  {
    // TokenState<'dimension'>
    dimension: (v) => { ... },
    // TokenState<'font'>
    font: (v) => { ... },
    // TokenState<'textStyle'>
    textStyle: (v) => { ... },
  },
  notMatched => { ... }
)

You can match as much type as you need, and if some are not matched, they’ll be caught by the callback in the 2nd argument.

If you're working with Typescript, you may face some type issues while using matchByType. A common example would be this:

tokenState.matchByType(
  {
    dimension: _ => 1,
    breakpoint: _ => 'hello',
  },
  _ => undefined,
)

If you copy/paste this code inside your editor, Typescript will complain and gives you the following error: Type '(_: TokenState<"breakpoint", Value, Mode>) => string' is not assignable to type '(token: TokenState<"breakpoint", Value, Mode>) => number | undefined'.

Basically Typescript is unhappy that we return a number and a string. To fix the issue, there's 2 solutions:

1. Return the same type:

tokenState.matchByType(
  {
    dimension: _ => 1.toString(),
    breakpoint: _ => 'hello',
  },
  _ => undefined,
)

1. Excplicitely set the return type:

tokenState.matchByType<string | number>(
  {
    dimension: _ => 1,
    breakpoint: _ => 'hello',
  },
  _ => undefined,
)

Depending on the use case, you'll prefer the first over the second solution, or the second over the first.

How to get the token value ?

Now that we know how to get the TokenState that we want, we want to extract the value from it. As mentioned before, a token can have a lot of places that can contains an alias, so when retrieving the value, there’s 2 ways of doing it.

Ignoring aliasing

If you only want to get the raw value of a token and ignore the aliases, you can use the following function:

tokenState.getJSONValue({
  resolveAliases: true,
  allowUnresolvable: false,
  targetMode: tokenState.modes[0],
})

Also, as combining matchByType and getJSONValue can become a bit heavy. For that case, you can use the following function:

tokenState.matchJSONValueByType(
  {
    // { unit: 'px', value: 12 }
    dimension: (v, mode) => { ... },
    // { family: 'MyFont', weight: 700, ... }
    font: (v, mode) => { ... },
    // { lineHeight: { unit: 'px', value: 24 }, ... }
    textStyle: (v, mode) => { ... },
  },
  notMatched => { ... }
)

Handling the aliases

Overview of the API

If you want to support the aliases in your output, you’ll need to handle them when they are some at the top level, mode level, or value level. To do so, we provide the Stateful Value API that helps you to make sure you handle all the possible cases. Let’s see an example and then break down everything.

// TokenState<'dimension'> -> { value: 12, unit: 'px' }
tokenState
  .getStatefulValueResult()
  .mapResolvableTopLevelAlias(alias => { ... })
  .mapUnresolvableTopLevelAlias(alias => { ... })
  .mapTopLevelValue(modeLevel =>
    modeLevel
      .mapResolvableModeLevelAlias((alias, mode) => { ... })
      .mapUnresolvableModeLevelAlias(_ => { ... })
      .mapRawValue((rawValue, mode) => 
        rawValue
	  .value
	  .mapPrimitiveValue(value => { ... })
	  .mapResolvableValueLevelAlias(alias => { ... })
	  .mapUnresolvableValueLevelAlias(alias => { ... })				
      )
      .unwrap(),
  )
  .unwrap();

Although this API is a bit verbose, it’ll make sure that you cannot miss a case. So, what’s happening?

When you retrieve the value of a token, we put it inside a StatefulResult , and it’ll expose the following type union at each level:

• Top level: ResolvableTopLevelAlias | UnresolvableTopLevelAlias | TopLevelValue

• Mode level: ResolvableModeLevelAlias | UnresolvableModeLevelAlias | RawValueSignature

• Value level: ResolvableValueLevelAlias | UnresolvableValueLevelAlias | string | number | ...

Each mapping function is dedicated to handling one case.

First, you’ll need to handle the top level:

tokenState
  .getStatefulValueResult()
  .mapResolvableTopLevelAlias(alias => { ... })
  .mapUnresolvableTopLevelAlias(alias => { ... })
  .mapTopLevelValue(modeLevel => { ... })
  .unwrap()

Note the unwrap at the end which is the way to extract the value.

Then you’ll want to handle the mode level:

.mapTopLevelValue(modeLevel =>
  modeLevel
    .mapResolvableModeLevelAlias((alias, mode) => { ... })
    .mapUnresolvableModeLevelAlias(_ => { ... })
    .mapRawValue((rawValue, mode) => { ... })
    .unwrap()

All you need to do for the mode level is to return a value, and it'll be mapped to the right mode.

And finally the value level (we take a dimension token as an example):

.mapRawValue((dimension, mode) => {
  const value = dimension
    .value
    // mapPrimitiveValue is not required if you only need the value inside
    // without updating it
    .mapPrimitiveValue(value => { ... })
    .mapUnresolvableValueLevelAlias(alias => { ... })
    .mapResolvableValueLevelAlias(alias => { ... })
    .unwrap()
    
    return value
})

If you call mapPrimitiveValue, it needs to be the first one to be called in order to avoid remapping results produced by mapUnresolvableValueLevelAlias or mapResolvableValueLevelAlias.

As you can see each step is always the same: handling unresolvable alias, resolvable alias and the value.

Resolving an alias

Sometimes, you need to use the precision of the stateful result API, but don't want to deal with the aliases at some levels. To avoid doing so, you can resolve an alias by calling resolveDeepValue, and you'll be left with handling the unresolvable alias and value case. Here is an example:

tokenState
  .getStatefulValueResult()
  .resolveDeepValue()
  .mapUnresolvableTopLevelAlias(alias => ...)
  .mapTopLevelValue(modeLevel => 
    modeLevel
      .resolveDeepValue()
      .mapUnresolvableModeLevelAlias(alias => ...)
      .mapRawValue((dimension, mode) => 
        dimension
          .value
          .resolveDeepValue()
          .mapUnresolvableValueLevelAlias(alias => ...)
          .unwrap()
      )
      .unwrap()
  )
  .unwrap()

Handling only the value case

Although we could get rid of the resolvable alias case, handling the unresolvable case might still be too verbose. So let me introduce you to a nice pattern to only deal with values:

1. We will check if the token is fully resolvable

2. We'll use resolveDeepValue to remove the need of handling the resolvable aliases

3. We'll use unwrapValue to remove the need of handling the unresolvable case at the type level

if (!tokenState.isFullyResolvable) {
  return
}

const output = tokenState
  .getStatefulValueResult()
  .resolveDeepValue()
  .mapTopLevelValue(modeLevel => 
    modeLevel
      .resolveDeepValue()
      .mapRawValue((dimension, mode) => 
        dimension
          .value
          .resolveDeepValue()
          .unwrapValue()
      )
      .unwrapValue()
  )
  .unwrapValue()

Thanks to this pattern, we went from a really verbose API to something lighter.