Benefits of Rules Driven Dynamic Forms

This post is written for decision markers on UI projects, typically architects or software leads and as well for front-end software developers. The topic is broadly applicable to different user interaction paradigms (Web, Mobile App, Chat, etc.…)

Writing forms is a very common activity. Simple forms are easy to handle with most UI frameworks; however, dynamic forms are difficult to develop and maintain and the difficulty typically increases exponentially.

What do I mean by dynamic forms? These are forms in which:

1. The total set of potential questions is large as is often the cases with many business processes nowadays. Of course, for good usability we only want to ask the relevant questions for the current user's case.
2. And, more interestingly, the set of questions to ask depends on answers from previous questions (previous steps) and/or from external data. This leads to independent questionnaire paths for various users.
3. Many times, the validation rules go beyond verifying a single input is correct. The validations rules can be complex and depend on previously entered data and external data.

For forms with a large number of questions and with dependence on previous answers we typically see the total number of independent paths increases (usually exponentially) and this quickly overwhelms the development team.  Programing all of these in code puts an enormous burden on the front-end development and testing teams and it is difficult to maintain and to scale up.

Additionally, it leads to difficulties reusing code across different use cases (different forms) and user interaction paradigms (for example, web app versus mobile app). 

To compound the problems, the business wants the questions to be changed relatively frequently. We wish we could give the business analysts who understand the use cases some tools so that they can maintain the questionnaires without code changes in the web front end and the mobile app. Additionally, for the mobile app, we would want a solution where we do not even need to request the user to update the app on their devices for any changes to the questionnaires.

To give you a concrete example, it’s typical for a travel insurance claim to have hundreds of different conditions and different paths in the process of collecting all the data for a claim. For example, in such use case, many questions need to be asked:

• There are several reasons for submitting an insurance claim ranging from trip was cancelled to lost luggage’s, injuries, sickness, ect.
• It can affect one or multiple persons on the trip.
• The questions may be different based on the policy the user has subscribed to.
• There are different questions and flows if the trip is cancelled before or after departure.
• We also need to collect various justification documents and expense claims.
• The validation rules on what can be entered are very complex and vary on inputs and previous data (for example, how much total amount is claimed in this claim and how much was already claimed in the year).

The list goes on and on. Below, is a visual representation of how the flow quickly becomes complex. And from which we can start to grasp the complexity of such use cases bring (and this diagram only scratches the surface):

One elegant solution to these problems is to use a rule system to define the model for the dynamic form independently of how it is rendered in the front-end UI and device. The model specifies what questions to ask at each step of the process and what paths the flow of questions take. It can specify simple to arbitrarily complex validation rules.

It’s a need I'm seeing in many industries that have complex processes, to name a few: healthcare, finance, education, state agencies and insurance. 

This is not new, various model/view design patterns have been used in front end development. So, the core design concept is very much familiar to UI teams.

What is new is the use of a rule system to define the model. This allows applying simple to complex logic to conditionally specify what controls to render, what validations to apply and what flow to follow. And as some of these rules systems are low-code, easy to use they allows business analysts familiar with the problem domain to create and most importantly maintain the model. 

The separation of concerns from the UI code responsible from rendering the questionnaire has multiple benefits:

• Organizations can deploy new questionnaires faster as this model enables creating client-side components (or renderers) that are reusable across different questionnaires.
• The model can run in process with the UI or can as easily be deployed as a remote service (Serverless on AWS, Azure or GCP) or as plain Node server. This is usually more convenient for Mobile App (IOS, Android) as it allows updating the model frequently without having to update the mobile app.
• Organizations can reach more constituents faster as the same model can to be used to drive dynamic questionnaires on different platforms because the model is abstract enough and can be reused to implement a UI rendering component on different platforms, for example on a mobile device and a desktop browser app.
• Reduce the load on IT organizations as the decision services can be created by business analysts that better understand the business problem. That is, business specialists can directly author the rules driving the forms and thus are able, for example, to add or easily change some questions without front end code changes.
• UI Front end developers can focus on the usability and not have to worry about the “business rules” driving which questions needs to be asked at each step.
• Doing UI refresh becomes much easier as the same model can be reused across different rendering UI.
• Testing is greatly improved as:

1. The model can be unit and system tested before integration. And the entire process can be automated and be part of a CI/CD pipeline.
2. Any change, by the business analysts can be validated by the business group.

In conclusion, if you are dealing with dynamic forms, consider using a rule system to model what questions to ask, what flow to take the user along and what validations to set. You will save lots of development time up front and you will also get more agility maintaining these forms. Check this solution at Github Dynamic Forms.