FME Dark Magic with SharedItem transformers

If you follow me on Twitter or read Mark Ireland 's weekly newsletter, you could see a couple of workspaces where I show some dark magic with SharedItem family of transformers. I would like to give you a bit deeper overview what I was doing there.

Changing T-shirt

With the changing t-shirt, Mark guessed it right - yes, I extract the raster from the model and replace it with another raster, which I bring into the workflow with FeatureReader. Note that in this case, we only need the raster, not the appearances or textures. A single raster can be used in multiple textures, appearances, and even geometries. So the workflow goes like this - we extract the raster with SharedItemRetriever, replace the original raster with the new one, and place it into the library instead of the original one with SharedItemAdder.

Reducing model size

I was asked to provide a few examples of the FME Flow AR apps, but the conditions was, the install of FME Flow shouldn't increase more than by 10 Mb. Packing three examples into 10 megabytes wasn't easy - 3D geometries and their textures can be pretty big. One of an easy steps here is to replace textures in PNG format with JPEGs, which compresses images with multitudes of colors a lot better. Check the video. My source data is in OBJ, the format that stores textures externally, that is, outside of the OBJ itself. The total size of all the rasters used as textures in the original model is over 3Mb. The trick here is to extract the rasters from the geometry, and then extract the raster into an attribute as a JPEG - we can do it with the RasterExtractor transformer. The format of the original raster does not matter - we can tell we want it as a binary attribute in one of the few popular formats from GeoTIFF to BMP, or, for a better compression in this case, JPEG. All we have to do after that is to replace the original raster geometry with our new compact format and return it back to the library. When features go directly from the reader to the writer, they check the library and under the same old ID, they find their raster, which is now a lot smaller. In my example, I make the size of the output FBX over 6 times smaller - almost 2.5 Mb without the magic vs 370 kb when I use it.

Lizard's new hoodie.

Here is one more example, which I add here mostly to show off Lizard's new clothing - now he, like many other Safers, has a new black hoodie, but honestly, I am not a big fan of black, so I tried to change it to some other colors. Let me know, which one you like more.

Again, all we have to do here, is to extract the raster, apply some raster algebra to it, and put the raster back - and by now you already know how :)

Important considerations

It is really important to know that SharedItem transformers have some limitations when we use feature caching. We need to always include both extracting and adding to the library transformers in a single run because every new run will assign new IDs to library items making previous assignments invalid. Usually, the transformation running only one of the SharedItem transformers (the second SharedItem in the flow) will fail, but under some circumstances, the workspace may work and produce unexpected results.

The order, in which transformers receive data, is also important. We need to send features to SharedItem first before they go to other transformers or writers so that the magic can really happen. Check "Set Connection Runtime Order" before SharedItem transformers to see and correct if necessary the flow of features downstream.

And a general advice for working with heavy 3D features - use Triangulator (and MeshMerger if necessary) to improve geometry visualization in Data Inspector.