Configurable Data Driven Approach towards Web Flows
Today’s world is a world where change is the only constant. Applications today don't only have to architect for delivering functionality but also have to architect to provision for change. In the context of web applications change is often related to change in the existing transitions/web-flows . Such changes often result in convoluted code which makes life miserable for everyone. The phenomena of frequent changes making every one's life miserable raises the question that in an environment of continuous change how do we architect our application to cope with change.
Handling Complex Flows:
So why do repeated changes in Web Flows for stateful applications create problems for everyone ? The answer can be found in the usual approach towards developing web applications which focuses more on integrating business logic with the user interface and tries to incorporate complex Web Flows as part of this integration. The fact that Web Flow integration requires special attention is quite often not realized and thus becomes the root cause of the failure of the application to respond to change . Lets look at an example to understand this phenomena better.
Consider an e-commerce site with a shopping cart. The customer logs in to the site, places his order, checks out and then logs out or places multiple orders. The flow diagram for the shopping cart is illustrated below
The above flow on the face of it looks simple. You define a login page. Once the customer logs in he starts adding products to his cart and checks out. A sample implementation based on Spring MVC is shown below
@Controller
public class ShoppingCartController{
@Autowired
private ShoppingCartService shoppingCartService;
/**
* <p>
* Check if the user is valid and create his session. Returns
* welcome view on successful login or error view in case of
* invalid username and password
* </p>
**/
@PostMapping("login")
public String login(@RequestBody User user,HttpServletRequest request){
LoggedInUser user = shoppingCartService.fetchUser(user);
if(user != null){
HttpSession session = request.getSession(false);
session.setAttribute("user",user);
if(shoppingCartService.pendingCheckout(user)){
return "checkout";
}
else{
return "addProductToCart";
}
}
else{
return "error";
}
}
/**
* <p>
* Capture details of product and the cart to which the product
* is being added in the @param ProductPurchaseDTO and
* add the product to the users cart
* </p>
**/
@PostMapping("addProductToCart")
public @ResponseBody String addProductToCart(@RequestBody ProductPurchaseDTO productPurchaseDTO,HttpSession session){
if(session != null && session.getAttribute("user") != null){
LoggedInUser user = (LoggedInUser)session.getAttribute("user");
return shoppingCartService.addProductToCart(productPurchaseDTO,user);
}
else{
return "error";
}
}
/**
* <p>
* Start the checkout process based on the user's cart. Once
* the checkout is completed take the user to the complete checkout
* view. If the checkout has failed then take the user to the
* error checkout view.
* </p>
**/
@PostMapping("checkoutCart")
public String checkOutCart(@RequestBody UserCartDTO userCartDTO){
if(session != null && session.getAttribute("user") != null){
shoppingCartService.checkoutCart(userCartDTO
, (LoggedInUser)session.getAttribute("user"));
return "completeCheckoutView";
}
else{
return "error";
}
}
Here typically we define separate end points for each functionality. For each functionality we add validations that ensures that the request for executing the desired functionality is valid. For instance a request for a checkout is only valid if the user making the request has a cart with unsold items. Similarly a user can only add products to a cart if he is logged in with a valid account and for a customer logging in, the system has to check whether the customer has any pending checkouts or not and redirect the customer accordingly.
Thus in the initial phase of the design a simple MVC architecture suffices and there does not seem to be an immediate need to focus on having a separate framework for handling the various Web Flows in the application. However once the Web Flows start to expand we very soon see that our current architecture is no longer sufficing. Since more than often such requirements need to be addressed quickly no one takes a step back to come up with a strategy to handle Web Flows but tries to incorporate the new flows in the existing architecture and as the frequency of these changes increase the existing code starts to become more and more convoluted. An example of this can be see when the flows in our Shopping Cart example change to return three separate views after a successful checkout by the Customer.
1) Normal Checkout: Applicable to first time customers or customers who are not frequent visitors and have just performed a checkout.
2) Checkout with a discount: Applicable to loyal customers who have just performed a checkout.
3) Checkout with an offer: Applicable to loyal customers during festivals who have just performed a checkout.
Below diagram gives a view on how the overall journey looks after the Checkout flow change:
In the aforementioned diagram the Checkout can either be a Checkout with Offers or a Checkout with Discounts or a Normal Checkout. Now this means that the Checkout endpoint,apart from performing the Checkout process, needs to have a logic to compute the next view corresponding to the type of Checkout the customer is eligible for. This change in the Checkout endpoint also needs to be reflected in the Login endpoint since the Login endpoint now has to direct a user during login to a customer specific Checkout screen for a customer that has a pending Checkout.
Incorporating this logic within the Login and Checkout endpoints starts to convolute the logic in the endpoints since now the end points apart from handling logic specific to their functionalities also need to determine the application state after execution of their functionalities. Below is a snapshot of how the code is changed after incorporating these flow changes
@Controller
public class ShoppingCartController{
@Autowiredprivate ShoppingCartService shoppingCartService;
/**
* <p>
* Check if the user is valid and create his session. Returns
* welcome view on successful login or error view in case of
* invalid username and password
* </p>
**/
@PostMapping("login")
public String login(@RequestBody User user,HttpServletRequest request){
LoggedInUser user = shoppingCartService.fetchUser(user);
if(user != null){
HttpSession session = request.getSession(false);
session.setAttribute("user",user);
if(shoppingCartService.pendingCheckout(user)){
// now determine the next checkout view based on the below conditionsif(shoppingCartService.isUserEligibleForDiscount(user)
&& shoppingCartService.isUserEligibleForOffer(user)){
return "discountOfferCheckoutView";
}
else if(shoppingCartService.isUserEligibleForDiscount(user)){
return "discountCheckoutView";
}
else if(shoppingCartService.isUserEligibleForOffer(user)){
return "offerCheckoutView";
}
else{
return "checkout";
}
}
else{
return "addProductToCart";
}
}
else{
return "error";
}
}
/**
* <p>
* Capture details of product and the cart to which the product
* is being added in the @param ProductPurchaseDTO and
* add the product to the users cart
* </p>
**/
@PostMapping("addProductToCart")
public @ResponseBody String addProductToCart(@RequestBody ProductPurchaseDTO productPurchaseDTO,HttpSession session){
if(session != null && session.getAttribute("user") != null){
LoggedInUser user = (LoggedInUser)session.getAttribute("user");
return shoppingCartService.addProductToCart(productPurchaseDTO,user);
}
else{
return "error";
}
}
/**
* <p>
* Start the checkout process based on the user's cart. Once
* the checkout is completed take the user to the complete checkout
* view. If the checkout has failed then take the user to the
* error checkout view.
* </p>
**/
@PostMapping("checkoutCart")
public String checkOutCart(@RequestBody UserCartDTO userCartDTO){
if(session != null && session.getAttribute("user") != null){
LoggedInUser user = (LoggedInUser)session.getAttribute("user");
// perform the checkout process
shoppingCartService.checkoutCart(userCartDTO
, user);
// now determine the checkout viewif(shoppingCartService.isUserEligibleForDiscount(user)
&& shoppingCartService.isUserEligibleForOffer(user)){
return "discountOfferCheckoutView";
}
else if(shoppingCartService.isUserEligibleForDiscount(user)){
return "discountCheckoutView";
}
else if(shoppingCartService.isUserEligibleForOffer(user)){
return "offerCheckoutView";
}
else{
return "checkout";
}
}
else{
return "error";
}
}
As more transition logic is incrementally incorporated into the main application logic very soon the code of the application starts becoming unmaintainable eventually resulting in production issues and unwarranted bugs. To avoid such a situation it is important to have a framework or structure for handling transitions.
Web Flow Frameworks:
To address the complexity that multiple web flows brought to a stateful application Web Flow frameworks started being developed. An example of a Java based Web Flow framework which started being used was the Spring Web Flow framework. Spring Web Flow focused on moving the transition logic out of Java Code into XML files. However very soon people saw that Spring Web Flow brought in its own complexity and was more suited for the conventional form navigation using Spring Views. Developers found that trying to use it for Single Page Applications and Ajax increased the complexity rather than reducing it and thus were weary of using it to model their Web Flows. This stackoverflow article https://stackoverflow.com/questions/29750720/what-are-the-spring-web-flow-advantages?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa talks in more detail about the problems of Spring Web Flow.
The reason why Spring Web Flow and Web Flow Frameworks like Spring Web Flow did not become the de-facto solution to this problem was the following
1) These frameworks tried to model a business process within the Web Application where the objective was to come up with a design which answers the simple question "If I am currently in state A then with my current data D which is the most appropriate state for me" .
2) Trying to bind the entire decision making process with Spring components. For example using Spring Web Flow only Spring Views or Spring Fragments could be rendered. However this introduced a rigidness which made it unusable for applications who were using a rendering logic different from the one used by Spring.
Given the problems with the existing approach the following points need to be kept in mind while designing an approach to simplify complex Web Flows .
1) The approach should be designed to try to answer at all times the question, "If I am currently in state A then with my current data D which is the most appropriate state for me". The approach should only look at two things, the current transition/state and the relevant data and all transition movements should be Data Driven and not Order Driven.
2) The approach should be configurable so that any changes to the transition logic would be configuration changes rather than code changes.
3) The approach should not tie its design to a particular view rendering strategy but should allow the application to decide a view rendering strategy for any transition change.
Keeping the above points in mind this article suggests a Configurable Data Driven Approach to resolve the complications introduced by multiple Web Flows.
Configurable Data Driven Approach to Web Flows:
To understand this approach we will be redesigning our Shopping Cart example. The below frameworks would be used in this example
1) Spring MVC: Java based MVC framework used to implement the MVC architecture on the server
2) MVEL: A hybrid dynamic/statically typed, embeddable Expression Language and runtime for the Java Platform.
The client side here could either be Angular JS, or normal jsp, or xhtml or even simple html . This approach will integrate well with any client side since it does not mandate any particular UI technology for rendering a view based on the transition.
As stated before we would like to separate the transition logic from the main code. For that we divide our server side code into 2 parts:
1) Generating Data: The MVC architecture would be responsible for generating data as a result of any action on the application. This data along with the current state would be used to decide the flow.
2) Deciding the flow: The part of the system determining the flow would have a mapping of all the possible transitions from a particular state. This mapping can be represented as a map with the key being the state and the value being a tree of Rules of all the possible transitions from the current state. Each rule in the tree would have an expression which would be bound to the data captured by MVC architecture and executed using MVEL.
The tree based on the rules specified on each transition would return one particular leaf node which would be the final transition. The below diagram remodels the shopping cart example taking the two functionalities which were effected by changes in the Checkout flow namely the Login and the Checkout functionality.
The above diagram is of the Rule Tree for the checkout process of the Shopping Cart example. Each of the transitions has a rule associated with it. Each rule checks whether a particular field has a particular value. For example the Checkout Process would only be initiated if the field userAction has the value (denoted by fieldData) of checkout. Similarly a Normal Checkout process should be initiated if the field "checkout" has the value (denoted by fieldData) "normal".
The expressions on each transition would be evaluated using MVEL and the MVC architecture would be feeding data to evaluate these expressions. The below code snippet re-designs the Shopping Cart example using the above approach
public class RuleBean{
private String transition;
private String rule;
private List<RuleBean> childRules;
//----------------getters and setters-----------------------------
}
public class Field{
private String fieldName;
private String fieldData;
//-------------- getters and setters-------------------------------------------
}
@Component
public class RulesComponent{
// this map of rules would be loaded from a persistent store keeping the rules configurable
private Map<String,List<RuleBean>> ruleMap = loadRules();
/**
* Load the transition rules for an application from a persistent store
* for each transition.
**/
public Map<String,List<RuleBean>> loadRules()
{
}
/**
* <p> Given a particular transition and data fetch the next data </p>
**/
public String fetchTransition(String currentTransition,List<Field> fieldList)
{
// get a list of rule beans corresponding
List<RuleBean> ruleBeanList = ruleMap.get(currentTransition);
// Pass a list of fields and rules to compute the list of fields
// with each rule for a match.
RuleBean ruleBean = fetchRuleBean(ruleBeanList,fieldList);
return ruleBean.getTransition();
}
/**
* <p>
* Given a data of fields and a list of rules find the rule which matches
* if the rules match and if the rule does not have any child rule
* else return the matched ruleBeans
* </p>
**/
private RuleBean fetchRuleBean(List<RuleBean> ruleBeanList,List<Field> fieldList)
{
for(RuleBean ruleBean : ruleBeanList)
{
boolean expIsTrue = false;
// iterate through the entire data and check for a match. Note that
// the tree structure ensures that siblings are mutually exclusive
// and hence the we only look for the first match
for(Field field: fieldList){
boolean expIsTrue = MVEL.evalToBoolean(ruleBean.getRule(),field);
if(expIsTrue)break;
}
// if there is a match
if(expIsTrue)
{
// if there are no child nodes return the matched nore
if(ruleBean.getChildRules() != null){
return ruleBean;
}
// in case of child nodes try to find the child nodes
else{
return fetchRuleBean(ruleBean.getChildRules(),fieldList);
}
}
}
}
}
The above code lays the outline of the Data Driven Rule Engine that will be used to compute the next transition from the current transition using the current data. Here the current data is denoted by a List of Fields each field with a name and value for the field. A field for any application will be the smallest unit of data. It is important that the smallest unit of data is used by the Rule Engine so that all possible transitions based on the permutations of all possible data points can be accurately modeled by the Rule Engine. With the addition of the Data Driven Rule Engine the controller code now does not need to incorporate any transition logic. Below is a snippet on how the Controller code changes specifically for the Checkout functionality.
@Controller
public class ShoppingCartController{
@Autowired
private ShoppingCartService shoppingCartService;
@Autowired
private RulesComponent ruleComponent;
/**
* <p>
* Check if the user is valid and create his session. Returns
* welcome view on successful login or error view in case of
* invalid username and password
* </p>
**/
@PostMapping("login")
public String login(@RequestBody User user,HttpServletRequest request){
LoggedInUser user = shoppingCartService.fetchUser(user);
if(user != null){
HttpSession session = request.getSession(false);
session.setAttribute("user",user);
if(shoppingCartService.pendingCheckout(user)){
if(shoppingCartService.isUserEligibleForDiscount(user)
&& shoppingCartService.isUserEligibleForOffer(user)){
return "discountOfferCheckoutView";
}
else if(shoppingCartService.isUserEligibleForDiscount(user)){
return "discountCheckoutView";
}
else if(shoppingCartService.isUserEligibleForOffer(user)){
return "offerCheckoutView";
}
else{
return "checkout";
}
}
else{
return "addProductToCart";
}
}
else{
return "error";
}
}
/**
* <p>
* Capture details of product and the cart to which the product
* is being added in the @param ProductPurchaseDTO and
* add the product to the users cart
* </p>
**/
@PostMapping("addProductToCart")
public @ResponseBody String addProductToCart(@RequestBody ProductPurchaseDTO productPurchaseDTO,HttpSession session){
if(session != null && session.getAttribute("user") != null){
LoggedInUser user = (LoggedInUser)session.getAttribute("user");
return shoppingCartService.addProductToCart(productPurchaseDTO,user);
}
else{
return "error";
}
}
/**
* <p>
* Start the checkout process based on the user's cart. Once
* the checkout is completed take the user to the complete checkout
* view. If the checkout has failed then take the user to the
* error checkout view.
* </p>
**/
@PostMapping("checkoutCart")
public String checkOutCart(@RequestBody UserCartDTO userCartDTO){
if(session != null && session.getAttribute("user") != null){
LoggedInUser user = (LoggedInUser)session.getAttribute("user");
// get all the checkout data based on the userCartDTO and user details in the form of a map
Map<String,Object> dataFields = shoppingCartService.getCheckoutData(userCartDTO,user);
List<Field> fieldList = new ArrayList<Field>();
for(Map.Entry<String,Object> dataFieldEntrySet : dataFields.entrySet())
{
Field field = new Field();
field.setFieldName(dataFieldEntrySet.getKey());
field.setFieldData(dataFieldEntrySet.getValue());
fieldList.add(field);
}
return ruleComponent.fetchTransition(userCartDTO.getCurrentTransition(),fieldList);
}
// return a fixed identifier indicating an error
else{
return "error";
}
}
All the if-else conditions have vanished. All that is required here is to get data from the ShoppingCartService, pass it to the rule engine in an agreed format and return the output from the rule engine. The only important thing to note here is that the current transition needs to be passed to the controller as all rules correspond to a particular current transition.
Similar approach can be used for the Login functionality. The Login functionality rule tree is displayed below.
The Login Rule Tree has 2 transitions. One to the Checkout Parent Node and another to the Add Product Node. The transition to the Checkout Parent Node will only be taken if for a particular customer the Checkout process is initiated, if not then the default transition would be the Add Product transition. Thus now the Login endpoint in the Shopping Cart Controller can be modified as below
@Controller
public class ShoppingCartController{
@Autowired
private ShoppingCartService shoppingCartService;
@Autowired
private RulesComponent ruleComponent;
/**
* <p>
* Check if the user is valid and create his session. Returns
* welcome view on successful login or error view in case of
* invalid username and password
* </p>
**/
@PostMapping("login")
public String login(@RequestBody User user,HttpServletRequest request){
LoggedInUser user = shoppingCartService.fetchUser(user);
if(user != null){
HttpSession session = request.getSession(false);
session.setAttribute("user",user);
Map<String,Object> dataFields = shoppingCartService.userData(user);
List<Field> fieldList = new ArrayList<Field>();
for(Map.Entry<String,Object> dataFieldEntrySet : dataFields.entrySet())
{
Field field = new Field();
field.setFieldName(dataFieldEntrySet.getKey());
field.setFieldData(dataFieldEntrySet.getValue());
fieldList.add(field);
}
String transtion = ruleComponent.fetchTransition(userCartDTO.getCurrentTransition(),fieldList);
if(transition == null){
transition = "addProduct";
}
return transition;
}
else{
return "error";
}
}
/**
* <p>
* Capture details of product and the cart to which the product
* is being added in the @param ProductPurchaseDTO and
* add the product to the users cart
* </p>
**/
@PostMapping("addProductToCart")
public @ResponseBody String addProductToCart(@RequestBody ProductPurchaseDTO productPurchaseDTO,HttpSession session){
if(session != null && session.getAttribute("user") != null){
LoggedInUser user = (LoggedInUser)session.getAttribute("user");
return shoppingCartService.addProductToCart(productPurchaseDTO,user);
}
else{
return "error";
}
}
/**
* <p>
* Start the checkout process based on the user's cart. Once
* the checkout is completed take the user to the complete checkout
* view. If the checkout has failed then take the user to the
* error checkout view.
* </p>
**/
@PostMapping("checkoutCart")
public String checkOutCart(@RequestBody UserCartDTO userCartDTO){
if(session != null && session.getAttribute("user") != null){
LoggedInUser user = (LoggedInUser)session.getAttribute("user");
// get all the checkout data based on the userCartDTO and user details in the form of a mapMap<String,Object> dataFields = shoppingCartService.getCheckoutData(userCartDTO,user);
List<Field> fieldList = new ArrayList<Field>();
for(Map.Entry<String,Object> dataFieldEntrySet : dataFields.entrySet())
{
Field field = new Field();
field.setFieldName(dataFieldEntrySet.getKey());
field.setFieldData(dataFieldEntrySet.getValue());
fieldList.add(field);
}
return ruleComponent.fetchTransition(userCartDTO.getCurrentTransition(),fieldList);
}
// return a fixed identifier indicating an error
else{
return "error";
}
}
As with the Checkout endpoint there is no if-else logic here . All that is done here is get the past state of the user, convert it into fields with each field having a name denoted by fieldName and data denoted by fieldData and pass these list of fields to the Data Driven Rule Engine. The rule engine takes care of all the branching and either returns a particular transition if there is a match or null if there is not match . If the Data Drive Rule Engine returns null then the end point returns the default transition which in this case is Add Product.
Note that in all cases the rule engine returns a string corresponding to a particular transition. However it does not mandate what that transition is but leaves it to the application to determine that. The application could map the transition returned by the Rule Engine to a template which would be loaded on the client side through a Angular JS Controller. The application could use the transition to render a Spring or html view. The string returned by the rule engine does not tie the view rendering to a particular rendering strategy but allows every application to define its own view rendering strategy .
Another point to note here is that the rules are loaded from a persistent store. This allows the rules to be stored in a database which then can be changed on the fly without having to re-deploy the application.
Conclusion:
In today's dynamically changing environment IT systems not only need to focus on delivery but also need to focus on provisioning for change. Creating a Data Driven Rule Engine is a way to provision for change by ensuring that complex transition rules which so convolutes the application code is replaced with a single call to a Configurable Data Driven Rule Engine which accepts a fixed set of arguments, returns the identifier for the next transition and ensures that any change in the transition logic is a configuration change rather than a code change. All convolutions and complexities are handled by the Rule Engine and the application needs to only focus only on particular actions rather than focusing on determining the next state.