Interface Developer Notes

2024-04-01 . Eduardo Sobrino

1.0 The Issue

While working with interfaces from despair systems issues will quickly be found just in how data entities have been recognized, conceptualized, and represented in each application/solution.  If each system to be interfaced was developed in any degree of isolation and the pressure to produced deliverables quickly for each presented use-case then it is probable that common data entities, data component patterns, and semantics shared by all of those were not taken into consideration.

2.0 Schema First or Manage Consequences

When using the (Developers favorite) Code-First approach, meaning that data components are defined as application/solution needs, them the effort may have not been guided by a unified data model or schema.  Data Architects usually prefer the Schema-First approach that requires the identification of all data components and the production of a data model sometimes before code is written.

When developing is recommended to review the newly introduce or modified data components with others (or the Data Architect if there is one).  The idea is to minimize the possible leftover complexities that later will require matching semantics, mapping data entities and/or elements, and the need to explain yet another introduced term in the model lexicon (vocabulary) later.

3.0 Interface Design Key Considerations

Data interfaces particularly targeting information exchanges are complex to build and maintain and for that reason several key points need to be addressed/considered including:

• Implement a data-mapping staging area based on a canonical (common) data model.  This will substantially improve the consistency of the mappings and keep terms in alignment with their true semantics.
• Discourage point to point interfaces.  In those the complexity increases as the number of needed interfaces increases (see Issue #1 and Solution exhibits ahead).  For example, in the worst-case scenario 6 point to point data exchanges require 6 (6 -1) = 30 interfaces, on the other hand using a common data model as suggested in the next point will require 6 2 = 12 interfaces.  As the number of systems increases the number of point-to-point interfaces to implement increases n * (n – 1) times (where n is the number of interfaces) to implement.

• Reduce the complexity of mapping data elements by performing the work on data transformation, cleansing, enrichment, and other procedures as close as possible to the source and/or target, meaning requiring mapping to a common canonical data model.  Point A (the source) map to the canonical model and point B (the target) from the canonical model.
• ·Resolve ambiguity close to the source and keep the unambiguous value in the canonical data-model.  Ambiguity is cause by overloading a field or column with values with different concepts/data-domains, for example a string data column called ItemDateText may contain an expiration date or the name of the producer.  Clearly the field/column is semantically overloaded.  The common data model may then have two fields one called ItemDate and the other ItemProducer.  Avoid other complexities introduced to handle those overloaded data elements by introducing a qualifier element stating how to interpret the ambiguous element.  Since reducing complexity is preferred avoid ambiguity as soon as it is found.
• The common (canonical) data model should use a suitable level of abstraction to have the flexibility of being applicable for many scenarios.  For example, while implementing a data component to manage Inspectors, Managers, Clients the term “participant” could be used to hold the ID of the person or organization in question and qualify the participant as type Manager, Inspector, or Client.
• Adopt a common vocabulary or create one (see related next topic).
• Create/Support a Canonical Reference Data repository meaning that a database with all needed Code Sets to manage the canonical code-description-category values should be available in or accessible to the staging area.  Different applications using the same code-set may implement the same concepts with different codes some numeric and others using string codes, also, each application may use different resolution meaning that will be more specific that others and therefore the code-sets don’t match.  For the previous a Reference Data repository is needed.
• The adoption of well recognized interface data models such as NIEM, OASIS, N-DEx, NODS, X11 EDI, HL7, FHIR or other also helps in adopting a common canonical vocabulary to map to and from.  This is useful if all exchange partners are willing to write and read from the common data model else the “information-sharing team” gets the burden to write those interfaces for them.

The previous is a partial list of things to have in mind while implementing interfaces/information exchanges.

Other things to consider:

• Implement an “Enterprise Service Bus” (ESB) if the volume and complexity of the exchanges is expected to be high and if reliable messaging is needed.
• If an ESB will not be available and as needed message queuing may help to manage large volume of data-exchanges reliably.
• Support the Publish and Subscribe messaging pattern, this will give others to subscribe to messages.
• Consider promoting the use of Web-Services/API’s.  Last year Microsoft released the API Builder that can help to quickly build-up a reliable API.  It may be possible to demonstrate that this tool can be used to build complex API’s with relatively ease.

4.0 Conclusions

Heist to produce data components may be a central issue on every enterprise.  Well, thought data models, lexicon, and easy to learn and use data assets is possible that will payoff thousandfold, most likely be easy to perform data governance in derive artifacts, and will definitively be AI ready.

If you are in the journey already and the design first was not possible as stated in the previous paragraph, just produce data components and artifacts that are consistent, have low or no complexity, resolve ambiguity early, and all the previous as soon as possible.

Constructive feedback and comments are welcome.