Automating Human Processes

I've been really fortunate in my new role at MELOS PUBLISHING. The founding team are highly experienced, supremely capable and very intelligent.

The business domain is, essentially, music royalty identification and collection - but with a focus on royalties due from TV advertising. This might sound very niche, but in 2019 $166.3bn was spent on TV advertising globally. Any music used in an advert incurs a royalty due to the composer and publisher of the music. If the brand in the ad happens to be one (or both) of those then that brand is due some royalties.

Now, a small percentage of a very large number is a large number. But, not only can MELOS demonstrably discover and recoup more royalties than any other similar business, but MELOS is putting its money where its mouth is in the form of advances. This can provide additional marketing budget to the brands but also, and arguably more importantly, it ensures the composers get compensated properly for the success of their works.

The Process

Prior to my joining the business, the team had developed and proven a process. Working with a large number of data sources, including data from brand partners, a lot of data analysis was taking place - but the majority of the "crunching" was being done by humans and took a significant amount of time. Various discrete platforms had been established to help, but in essence this was all a proof of concept and was not scalable.

Breaking down this large process into manageable, automatable components was going to be a challenge.

• The data is large, non-homogeneous (despite largely being of the same purpose) and frequently erroneous.
• The data can be retroactively amended.
• The data is largely time-series but massively inconsistent in regards to timezones, aggregation, etc
• Key referential fields are often human derived strings
• The processes are highly iterative and have benefitted from human insight.
• The business is growing fast - signing up one major new brand every couple of months. On-boarding new clients can take a few months due to the amount of discovery that goes on which can not be automated.
• It's a small team. We have to be adaptable - fitting our technical strategy around the daily distractions.
• This is a huge domain with lots to learn. We're all learning every day. We have to adapt our technical strategy frequently.

So what... this is normal right?

For the most part, in my humble experience, this is pretty normal for a young, tech startup. I've always enjoyed developing the engineering architectures and components with the team - then getting my hands dirty in the build and implementation. Fitting this around growth, steep learning curves and daily distractions is "normal".

However, if this was easy, everyone would be doing it. And the MELOS service does not have many competitors. We're not aware of anyone getting the success rate we're getting. That results in a lot of pressure on a tech team. We can't deliver any solution that performs worse than the legacy process.

Automating in stages

The architectural paradigm we decided to implement was one of a Service Oriented Architecture (SOA) but with some of those services being micro-services to help us break the domain in manageable components. A small team needs to be able to focus on delivering tangible benefits quickly. Delivering a component via an API that removes a formerly tedious or slow process is tangible!

We built a Data Lake to capture all our data as soon as it arrives. We broke down the human processes into a pipeline and discussed it with the entire business. We identified the core steps that need to be undertaken as we move data through the pipeline. And, we introduced a common business domain language so that, at least internally, we could all discuss and understand important topics efficiently.

Today we're completing some of the most important components in our new pipeline. Without all the pain and iteration performed by the team before (and since) my joining we wouldn't be here today. We've already got a strategy for 2021 and are excited by the prospects.

A bit thank you to everyone at MELOS. This is truly a team effort and I am humbled to work with such an enthusiastic and supportive team of industry experts.