Automated FME workspace documentation with FME

It is my conviction that FME is a very sophisticated tool to manage and manipulate data streams, in order to get the information we need at the end. Due to the endless ways to get things done with FME it is not always clear in what way the information has evolved. From this point of view it is obvious we want more order and a better understanding of that evolution.

Unfortunately describing what is happening in a workspace isn’t easy. It is time-consuming the task to try and explain what is happening to the data stream when using the highly configurable transformers and their behavior. An extra challenge therefore is a change in the workspace which should result in a change in the already finished documentation. Very recognizable is the effect when a manager askes where the documentation of the workspaces is? Most of the time this is the moment that there has to be thought about the state of mind we had when we created the solution and why we made the choices we did, and that is exactly what we want to preserve. It mostly never goes about the how but almost always about the why.

My fundamental idea is to make the documentation a part of the workspace and transformers itself, just like within other programming languages. Therefore we use bookmarks, bookmark descriptions and annotations with explanatory text. Through these facilities we can write down our decision making arguments and we can then reproduce our thinking about the choices we made. Putting numbers in the bookmarks and even nested bookmarks we can clarify the way the data moves through our process in the workspace. In combination with a properly filled workspace properties box, we have all the ingredients necessary to generate a describing document of the workspace.

Because with the use of the FMW reader we can generate output about a workspace, we can extract text and numbering from the bookmarks and workspace properties. The information we write to the output can have the workflow order according to the numbering in the bookmarks. This way we can automatically generate documentation about the working of a workspace which we can share with college’s or the system administrator on a wiki site for example. Because the documentation can be generated automatically with each change, it is always accurate and up to date preserving the quality of the work. The documentation produced by this process is very structured and therefore easy to use interpreted. This documenting system is easy to use, reuse, maintain and share with other FME users.

This primitive workspace shows a little of what the possibilities are. The result is shown below:

1: Reads all bookmarks from the selected workspace

2: Get the consecutive number of the text in the bookmark

3: Sort the bookmarks in a list on the consecutive number

4: Concatenate the text from the bookmarklist

5: Write the bookmarktext to a file

If you like to know more don't be shy and contact me.