Computer Assisted Compliance

The Engineering Process

For 'physical' products, not software only, many physical products include software anyhow, the design engineering process looks a little like this:

Considerable improvement effort has gone into many stages of the process, engineers and scientists have been using computer since their very beginning (H-bomb simulations!) to assist with every aspect of the engineering process.

There are established, mature, powerful and sophisticated tools for many aspects

• Generative Design
• Computer Aided Design
• Simulation (FEA/CFD)
• Product Lifecycle management (PLM)
• Even, Requirement Management

Computer aided, or assisted, engineering, CAE, is not new, it is ubiquitous. 

Then Machine Learning (ML) got useful. The recent developments (the combination of better algorithms, more computer readable data, and more powerful computers) means computers can now access new areas of the development process. Compliance?

Compliance

What is compliance (Certification, validation or qualification)? It has many names, but it basically the same; demonstrate the product does what it is meant to do.

Coming from an aerospace background this is REALLY important, a LOT of effort goes into demonstrate the compliance of parts, often MORE than the actual design of the product. Really.

The compliance process is complex, and there are whole fields dedicated to it. On the simplest level there are 4 stages, gathering requirements, defining the MOC's or methods of compliance, creating the required evidence to validate the requirement is met.

In reality it is a complex web of test plans, specifications and reports. There are already many systems to help manage this, in a 'database' type fashion, but not the actual compliance assessment.

So how can computers help?

Computer Assisted Compliance

When trying to demonstrate compliance there are two questions?

Before development, given some requirements what are the best Methods of Compliance (test standard, analysis etc,.) and what sort of evidence do they required.  

After development; I have some evidence and a requirement - am I compliant?

So CAS will need to address these sort of questions. Now, which ML, computers are getting better are addressing complex questions. From Siri to financial trading.

GPT

GPT is an awesome piece of software engineering, beyond the scope of article, this is a good introduction https://towardsdatascience.com/gpt-3-101-a-brief-introduction-5c9d773a2354

On the highest level GPT is a massive sophisticated TRANSFORMER - take 'input' and transform it 'output'. The clever thing about it is, with the amount of knowledge it has, it is capable of predicting output for a given input. 

For example:

• INPUT: It is a sunny and hot summer day, so I am planning to go to the…
• PREDICTED OUTPUT: It is a sunny and hot summer day, so I am planning to go to the beach.

The latest GTP, GPT3, is massive - like 175 billion parameters massive; the English-language Wikipedia, spanning some 6 million articles, makes up only 0.6 percent of its training data. 

GPT is based around language therefore it good at handle Certification and qualification type requirements that are, well, written down language.

Like most modern neural network models they are 'pre-trained' but allow some level of 'Fine-Tuning'. Which means the GPT can be 'tuned' with applicable Certification and qualification document data to 'tweak' the neural network model weight and bias toward that domain knowledge while still keeping all the language inferences knowledge.

So once its been 'trained' with a database of existing compliance documents, if provided with a requirement if PREDICTS the evidence needed, or given the requirement and evidence it PREDICTS if it is compliant.

Now these 'predictions' are based on the inherent natural language understanding of GPT together with the domain 'fine tuning' 

Now that may seam strange, but if we changed PREDICTs to evaluate that is exactly what a human does when assessing qualification data. The computer just does it quicker and more consistently.

Proof of concept

A proof of concept, using a 'toy' mini GPT, version, based in GPT2, is starting to be developed. 

The proof of concept is tiny compared to the developmental production solution, based on GPT3 and a much larger certification database, but still took a lot of computing power to train, and even infer (predict) takes a good slug of processing power. GPT3 takes even more computing. All this computation and development and hassle is, of course, invisible to the end user, which only has to wait a few seconds for a response.

However, the system is pretty simple to use; just gives the user some selection check boxs (for scope selection) and a text input field in which to ask the questions.

First example one; PRE compliance

• INPUT: Aircraft seat certification
• PREDICTED OUTPUT: Aircraft seats certification required to withstand a 16g dynamic force. A 16g seat is tested in a manner that simulates the loads that could be expected in an impact-survivable accident. Two separate dynamic tests are conducted to simulate two different accident scenarios: one in which the forces are predominantly in the vertical downward direction and one in which the forces are predominantly in the longitudinal forward direction......

Not bad - the 'toy' demo had focused its training on aircraft cabin interiors compliance. Why interiors? The idea being the application of this sort of new technology would be best suited to a highly defined and regulated, but innovative, sector.

Example Two; POST compliance

• INPUT: Aircraft seat 14g test 1300lb lumber load
• PREDICTED OUTPUT: For part 25 aircraft, Test 1 is a primarily vertical impact test with the characteristics of a minimum impact velocity of 35 fps with peak acceleration of 14 G’s and an impact angle of 30 degrees off vertical. Of particular importance is the requirement that during Test 1, the peak compressive lumbar load in a Hybrid II ATD, or equivalent, must be below 1500 lb.

Not perfect, but certainly useful.

Conclusion?

It certainly grasps natural language, both input and response.

There are certainly questions around scope capability. The 'toy' CAC responses where from a highly defined, highly regulated, but very narrow, product range; like Aircraft Interiors.

The larger development will take the production system into other, less defined, compliance regimes.

But, maybe, with the large amount of complex, regulatory, compliance requirements there is a benefit for a little computer assistance, like some much of the engineering process.