Build efficient Salesforce Applications with large Transactions and Data.

In this fast world, Enterprises push to get the application live in market with an acceptable performance and then later plan the performance improvement. Though this is completely reasonable from business perspective, technical team should keep scope for any performance improvement activity to be carried out later. I understand from my experience that any architectural change later will take a lot of effort. In the worst case it may demand rewriting the complete application. This may not be the case with programming improvements, those are more manageable and can be taken up later without big impact on the application. 

Large transactions and data bring a lot of complexities that influence the design of the application you are building on Salesforce Platform. The complexity lies in establishing peace and equilibrium between the business requirements and respecting salesforce limits, supporting desired response time and system throughput etc. In this article, I want to briefly discuss the architecture you can follow to design efficient and high performance application to support such complex scenarios.

An efficient design and code should keep the system scalable, increase data and transaction capacity by optimizing the transactions to run fast while utilizing less system resources. Practically speaking sometimes the design may not truely conform with other programming requirements like maintainability, coding best practices and may not leverage out of the box programming features that Salesforce provides. But always remember, the ultimate goal is to improve the performance of the application and other requirements are important but secondary.

The Setup

For the sake of this discussion, assume you are building a Salesforce application that manages Lead to support Sales. You have a team that uses this application to submit leads they gather from their field work. You have other systems in place that receive Leads from other sources and feed it in one-to-one or Bulk mode to this Salesforce app via integrations. The Salesforce app would need following interfaces to get the Lead data from these described sources -

Figure 1 : Application Interfaces

• API Interface - The Salesforce app exposes custom Apex REST API to integrate with let say a native mobile app that submits a Lead record for every user inquiry (one lead in one API request). Assume your native mobile app submits leads to Salesforce app in high volume.
• User Interface - Sales team that logins to Salesforce app and submits leads from application user interface.
• Bulk Interface - Systems that gather data from various sources like partners and data providers and submit leads in bulk to Salesforce app.

Design Solution

The proposed solution design is to have a common service layer that serves all the data interfaces and has only the operations common for all interface. Each interface has its own service layer and contains business logic specific to that data interface. In this way no data interface slows down the other interface with its specific operation.

Figure 2 : Solution Design

The Salesforce application architecture is described briefly below -

• User interface can be custom ( VF/Lightning) with most of the business logic in the apex controller, thus its user interface specific operations don't run for API transactions and bulk transactions. The apex controller will serve as it own specific layer.
• Bulk operations can have staging, and all the processing of the data can be performed on the staging object and pre-processed data can be inserted to lead.
• API transactions can have their own processing layer where the JSON/XML payload from API is processed and committed to database.
• Trigger/ Trigger handler - Serves as a common layer, performing only the operations common for each interface. This would processes the data in right amount and does not add any computation overhead to any of the data interface.

This architecture would not add any computation overhead to any interface and thus make it more scalable when your application supports high volume API and has large data volume.

Notice the use of custom user interface and the business logic in the apex controller instead of Salesforce standard user interface . With standard user interface, most of the business logic needed lands in the constructs like workflows, process builders, validation rules and triggers. This is good if user interface was the only data interface for the Salesforce app. These constructs run and get evaluated for every API request that creates or updates a Lead. Their evaluation criteria may not evaluate to true, but believe me, just checking if their criteria evaluates to true is a computationally intensive operation. And this can be a major setback to the performance if the number of these construct are more in number. These also affect the bulk insert or update transactions in similar manner.

I worked on reengineering an application that had 65 workflows and process builder on Lead object, which used to run on every insert or update of Lead. And a big part of the user interface business logic was coded in the trigger. These were so heavy that just evaluating if they meet the criteria, took 2 -3 seconds per DML operation( i.e. Per API Request that performs a DML on Lead). Again, most of processing was not needed for API transaction but for User interface. Since, we are discussing this setup in the context of high volume API and large data volume, these service delays and performance degradation are quite significant.

On the similar lines, if the application has a staging to support the bulk operation, all the processing specifically needed for the bulk can be performed there without being slowed down with processing overheads of other interfaces. The staging can then move the data to Lead for the rest of the processing.

You may get lured for using these out of the box programming constructs to save development time and reduce code maintenance or build a generic layer to support to all three interfaces but again do not forget, it is question of horses for courses !!

That is for now and stay tuned for the other article in this series !

Hello There ! I am Tohar. I can understand dog barks and cat purrs :) Also, I have extensive experience in leading multiple projects on Salesforce platform. I expertise in database and application design.

Please feel free to reach me on linkedIn, If you have any question or want to help me correct any bit in this article. All views expressed here are own and the article is only for knowledge sharing purpose.