Analytical architecture evolution - part 2
Before we dive into the ecosystem there are two key concepts to discuss:
- Loose coupling; and
- Metadata-driven
Loose coupling is an approach to connecting the components in an ecosystem in such a way as to ensure the components depend on each other to the least extent practicable. This isolation ensures that we can make changes in each component independently from all other components.
The issue is that traditional applications are not really built to support loose coupling. In the previous example I gave in Part 1 the CRM application is tightly coupled to the CRM datamart, the CRM datamart is tightly coupled to other data structures in the relational database and so on. You need to make a judgement call - if you place a high value on this concept then you’ll probably need to build your own applications.
The other concept relates to configurability – to deliver maximum agility a great approach is to think about everything being metadata-driven. This ranges from data pipelines, feature evaluation (more about this in Part 3!), campaign reporting and even predictive models.
If we think back to the example in Part 1, the CRM datamart was probably created by a script or ETL job with zero configurability, for example if there was a transformation (e.g. to create a segmentation) this would be typically ‘hard-wired’ instead of being driven by metadata.
Ok, so now that we’ve got these two concepts, I’ll list out some of the components of the ecosystem required to deliver marketing campaigns:
- Campaign Brief (to facilitate the capture of information relating to the campaign)
- Feature Store (to build features relating to the entity you care about, i.e. customer)
- Campaign Application (to process and ship the campaign – linked to channels)
- Campaign Reporting (to report on the campaign and close the loop)
There are other components required as we become more sophisticated, but let’s consider this the engine room!
In Part 3 I’ll spend a bit of time on the Feature Store…
very interesting mr hunter. thoughts on if the customer management application youre using doesn't have easy configurability and its dependent on the datamart? (ive worked with sas marketing automation recently and the infomap seems hard wired as you put it)