Are we worrying about the wrong Data Security challenges?

I regularly read articles on the latest worries about Data Security and Data Integrity (not necessarily the same thing) and while these are all key, I do wonder if they are actually the weakest links about which the CDO/CIO/CISOs should be thinking?

Thinking on my roles over the last decade I have worked in various companies to deliver technologies to make certain tasks easier; be it the code protection of individual executables, the delivery of secure, easy to use Payment or DRM SDKs or the reduction of effort in the deployment and integration of Advanced Data Models by 10,000%.

Most large organisations are (understandably, and usually correctly) relying on many security layers protecting their data within their systems. There is a huge and very profitable industry that has grown up around software and hardware technologies designed to protect the integrity of data and the information stored on servers, on-premise and in the cloud. There is a reasonable argument that given the focus and sheer magnitude of cloud providers, these may even be more secure than a company’s own servers, housed in their own buildings. 

But we still see breaches. Un-protected AWS buckets, forgotten servers, Apps rushed to production & poorly pen-tested (or not tested at all) and APIs assumed to be secure but with no authentication in place. Technology to solve these issues exist and have existed across multiple markets for many years. Some of them even work.

We’re now moving into a new world of Data, the growth in this market has meant that Data Scientists are among the most sought-after (and highly paid) “software developers” of any kind. Ask many corporate CEOs (or Health Ministers) today what is one of their key assets and they will all answer “Data, and the Data/AI/ML Models that consume it”.

And yet the individual unique applications that are created, for that is what a model is, are delivered to the cloud or to IT systems with little or no thought to security. 

Compiled languages like C++ can be easily de-compiled and run-time compiled languages like Java are easy prey to even the simplest high school hackers using tools downloaded from the web. Code written in these languages can be analysed like a film editor can run a sequence frame by frame, and like a film editor, hackers can cut and paste new malicious code into the application or remove other code as required.

While in the very best cases the data centres have barbed wire around them, their hardware and software firewalls are in place and the encryption keys are agreed, all protected by multi-factor authentication. While user management KYC and the rest are all part of the story, the APIs to the outside world are secured, has anyone thought to secure the source code of the models being delivered to production? 

I mean, why go to the bother of subverting the data when you can compromise a model before or during deployment in order to subvert the information you need or deliver their payload?

What price AI/ML Model Integrity?