The pertinent question to ask is whether sensitivity analysis is a tick-mark approach or does it serve a wider purpose. I have practically seen it being used as a tick-mark approach with very limited commentary about the interpretation of the results. If you are a model risk management practitioner, irrespective of the line of defence you are working in, you might find these questions handy in critically evaluating the results of the sensitivity analysis.
- This one is very interesting, and I have seen audits/regulators questioning this part in recent times. How do you calibrate the shock size in a sensitivity analysis? For example, are you using mean +- 1/2/3/… standard deviation shocks? Does the choice of the shock size in a sensitivity analysis linked with the model purpose? Can that choice be calibrated to the historical data and hence made objective?
- From a given model specification of certain typically used algorithms, e.g., multiple linear regression, binary logistic regression, etc., can’t one make it out the expected impact of change in one of the independent variables on the model outcome, ceteris paribus? Then, what is the need for fancy charts and paragraphs in the model development document around sensitivity analysis? Or is it that the sensitivity analysis is more meaningful for certain complex algorithms?
- Can sensitivity analysis be used as a tool to determine the operating boundaries under which the performance of a model is expected to be acceptable? Well, as a model validator, I have used sensitivity analysis to highlight the model design issues and made it impactful. Have you?
- Is sensitivity analysis done in a comprehensive manner, i.e., by evaluating a model’s output by shocking the model inputs, parameters and/or key assumptions made in the model development process or is it solely focusing on shocking the model parameters only?