API Pattern of the Week: Metadata Element

Our API pattern language is divided into several categories. This week, we'll revisit the Structure category, from which we've already discussed Error Report and API Key. The patterns in this category address the following design issues: What is an adequate number of API message parameters? How should these parameters be structured? How can they be grouped and annotated with usage instructions? So let's have a look at the Metadata Element pattern.

Context

The request and response message representations of an API operation have been defined, possibly using one or more of the basic structural patterns (Atomic Parameter, Atomic Parameter List, Parameter Tree, and Parameter Forest). To process these representations accurately and efficiently, message receivers require more information about their meaning and content than just name and type.

Problem

How can messages be enriched with additional information so that receivers can interpret the message content correctly, without having to hardcode assumptions about the data semantics?

Forces

This design problem is discussed in terms of the following conflicting forces: Interoperability, coupling, and ease of use versus runtime efficiency (message size)

Solution

Introduce one or more Metadata Elements to explain and enhance the other representation elements that appear in request and response messages. Populate the values of the Metadata Elements thoroughly and consistently; process them to steer interoperable, efficient message consumption and processing.

Figure 1 sketches this solution (PlantUML, from pre-book times):

Three variants of this pattern exist, each representing a particular type of metadata: Aggregated Metadata Element, Control Metadata Element, and Provenance Metadata Element.

Example

The following example shows examples of three metadata types, provenance (regionCode), control (nextPageToken, prevPageToken), and aggregated metadata (totalResults, resultsPerPage), all organized as Atomic Parameters. The pageInfo forms a simple Parameter Tree that bundles two of these.

GET https://www.googleapis.com/youtube/v3/search?
    HSR&part=snippet&maxResults=2&pageToken=CAIQAA&key=<KEY>

{
 "kind": "youtube#searchListResponse",
 "etag": etag,
 "nextPageToken": "CBQQAA",
 "prevPageToken": "CAoQAQ",
 "regionCode": "CH",
 "pageInfo": {
    "totalResults": 1662,
    "resultsPerPage": 10
  },
 "items": [
     search Resource
  ]
}        

The HTTP concept of etag, defined in RFC 7232, qualifies as control and provenance metadata.

Discussion

The resolution of the pattern forces is discussed in the book.

More Information

The Pagination pattern relies on metadata to inform the client about the current, previous, and next result pages and the total amount of pages/results, etc. HATEOAS Link Elements (typed link relations) contain metadata as well.

Metadata is covered in the information management literature. Two examples are Introduction to Metadata, 3rd edition (Baca, Murtha, ed., Getty Publications 2016) and A Gentle Introduction to Metadata by Jeff Good.

The Zalando RESTful API and Event Scheme Guidelines point out the importance of OpenAPI metadata.

In the context of distributed hypermedia systems and Web APIs, metadata is mentioned and characterized in the formal definition of REST as an architectural style; a blog post by Steve Klabnik covers metadata in resource representations.

The "API Pattern of the Week" series here on LinkedIn featured the following patterns: Wish List, Pagination, Error Report, API Key, Processing Resource, Information Holder Resource, Data Transfer Resource, and State Transition Operation.

Find the complete pattern description in Chapter 6 of "Patterns for API Design," for instance, at InformIT. There is a 35% discount on the book with code API-PATTERNS.