Secure Testing of Software in Cloud Based Subsystem - iPST

Introduction

The US Government obligates software vendors to deliver a percentage of a code on each project as Open Source. Sounds counter-intuitive? There is more to come. Physical separation did not prevail as a dominant strategy in preventing information leaks. Deep wounds were inflicted but strategy remained the same: go head on rather than hide. The best strategy in Karate is not blocking but not being there.

There is understandable resistance in the industry against testing in the production environments.

The red flags and risks are multiple:

1.           The very idea means that the software was not tested earlier elsewhere

2.           Even if logically correct a new software might crush the entire system due to unforeseen side effects.

3.           The testers and developers get a glimpse of ‘real data’ that could easily constitute a breach of regulatory rules or harm business due to revealing some secrets.

Yet, government pushes for transparency over secrecy.

Let’s try to have a quick look into some of these ideas because they come from the world where security cannot be compromised even if higher needs ought to be catered.

A randomly scheduled, a randomly targeted audit of decisions performed by workers and by the system itself is one of the activities commonly used by government-run systems..

Such an audit requires a set of credentials with very high privileges. Auditors, if humans need to have high clearance. If humans, that is it!! What if the entire process of auditing is scripted and final decision makers do get all they need without SSN and other PII data?

Why does average institution, many folds smaller would care for the above? What problem am I proposing to solve?

Most business systems, such as CRM, based on Salesforce, SAP, Zoho, Monday, HubSpot, … you name it, take data from and send data to several other equally complete, complex and functionally self-contained systems. 

Problem:

Testing of the system that takes data from other systems that dwell beyond the integrator's control became a new challenge.

The major players can afford themselves emulators, simulations and so on. But these constructed by the integrator risk being flawed and not up to date; and these provided by the source (like CMS for example) are rare.

Proposed Solution, the in-Production System Tests (iPST):

1.           Create a subsystem in the delivered product that is capable of:

a.           Intercepting a portion of the incoming data and:

i.            Forking a scrambled PII copy to Business Objects to be used by Audit User

ii.           Picking data according to specific characteristics (the test data)

b.           Creating a set of short-lived credentials for Audit Users

c.           Triggering a Test Script execution once enough test data is gathered.

2.           Provide Test Data selection criteria specific to every inbound data Stream and relevant Test Script.

Note that in lower environments the system must guarantee that the original inbound stream is destroyed and only PII purged copy of data is allowed to live.

Once test/audit is ended the Test Data needs to be deleted, the business objects created by Audit Users need to be deleted, the entire subsystem of intercepting and forking a portion of the data needs to be inactivated, and finally the Audit Users need to be inactivated.

The lower environment in the above context would be some sort of PRE-PROD, as opposed to DEV, QA or INTG.

Note that some sources will contractually, or on grounds of higher regulations will prohibit you from data stream splitting.

Also, the above points are outlined to present the idea. The actual implementation might require that Audit Users create Business Objects before these accept traffic from inbound stream(s).

Finally, these tests are intended to be completed in the PROD environment before the system is handed over to the end users for good. However, they are intended, as well, to be used as part of regression set many times after.

Your feedback is truly appreciated.