Impact of Software Design Considerations

— A Case Study

How well a software system serves its purpose depends predominantly on what factors were taken into consideration while designing the software.

This informal case study is my introspection of my own approach while designing one of my significant pieces of creation: a middleware framework for creating enterprise integration solutions. Far from being an example of perfection, this is just a look at what all I considered in specific context scenarios and what results it produced.

The Context

The goal of creating this framework was to be able to create customized enterprise integration solutions. The target industries were quite widespread and included telecom, finance, energy, utilities and manufacturing.

It was often required to show a quick demo of the customized features proposed to the client. This required the capability to quickly build the actual customized feature.

As we had adopted an agile strategy for our technology professional services, we were also required to be able to deliver incremental working builds to the client. This was so that the client regularly receives working progress of the system. It also enabled them to make changes in their requirements and expectations based on actually seeing and working with the developed and delivered features.

The solutions were often required to be deployed for large volume of real time processing, that would further increase manifold with passage of time. This required the solution to be highly scalable as well. To simplify, ease and empower the client’s systems and infrastructure administration teams, our solutions were also required to be easily scalable, easily deployable, and highly and easily configurable.

The business solution being addressed was often mission critical and required the systems to be extremely robust and fault tolerant. It was required that whenever any problem arises, it should be resolved within hours for critical issues. This required that we should always be able to diagnose the problem immediately and be able to fix the problem immediately as well.

Due to the inherent nature of an integration solution, the solutions required to be able to provide and consume services in a service oriented architecture. This applied to our solution’s interactions with any other systems and applications within the client’s infrastructure, regardless of their operating system, technology platform, programming language, or development environment. Moreover, the solutions also needed to be flexible enough to be able to communicate with any channel or medium or end point (whether email, fax, sms gateway, messaging systems, any kind of web service, web application, mobile apps of any platform), interface with any database, any active directory structure or LDAP server, any other third party integration platform, ERP, ECM, CRM, or workflow system.

The Design Choices

Considering all of the context, and fortunately having the liberty to make my choices as long as the end result met the business goals well, I proceeded to make my design choices, in a very organic manner.

I decided to build own framework from scratch instead of depending upon a third party integration platform or an application server platform. The reasons were:

1. A third party platform would add significant additional cost to each solution we would sell, reducing profit margins severely.
2. Many of our clients would not be willing to take the big step of embracing our proposed third party integration platform or application server platform just to incorporate our proposed business solution into their infrastructure. It would only make sense for them if they decided to onboard their other systems and applications onto that platform as well. So our target clients would be reduced significantly.
3. A third party platform product, no matter how comprehensive or amazing it is, along with its strengths, would also bring with it its own limitations, restrictions and dependencies, over which we won’t have any control. Whatever solutions we propose based on third party platform product, our solution would always be ridden with those additional limitations, restrictions and dependencies. These could be limitations in numbers, types or ways of possible integrations or possible communication channels. It could be a dependency of upgrade or migration of the whole system. It could be dependency on just the technical support of the product provider for any issues with their propriety product that could be sabotaging our solution and the client’s business. It could be performance issue due to the technology choice of their product.

I chose to use Microsoft technologies for the framework.

• Microsoft Visual Studio as the integrated development environment (IDE).
• C# as the programming language.
• Microsoft .NET as the development framework and platform.
• Microsoft SQL Server as the database.
• IIS as the web server.

These choices transpired because:

1. All of our enterprise clients already had Microsoft based infrastructures and licensed software. Moreover, we already had Microsoft licensed technology to do our development. So it minimized additional cost, and leveraged already existing technology assets.
2. I was already familiar with these technologies, so it reduced learning curve and solution delivery time.
3. Microsoft technologies are supported by a huge online community. Any problem you face is most likely already faced and resolved and documented online by others along with best practices and pitfalls. This hugely reduced the time spent on resolving issues, fixing bugs. It also minimizes the chances of getting stuck with a problem that you cannot resolve (at all or quickly), e.g. an issue or restriction in propriety software.
4. Microsoft technologies enabled us to create and run native code on native platform. This enabled the performance required for high volume and real time processing.

I built the core of the framework using windows services and web services. This enabled the system to be:

• Highly and easily scalable, and easily deployable.
• Service oriented, able to provide services to and consume services from other systems regardless of their platform, development environment or operating system.
• Incorporated with built in multithreaded, multiprocess high concurrent performance capabilities.

Having taken agile approach, I considered the system to be always a work in progress. So the system was always improving at every opportunity, yet always production worthy.

A centrally important design choice was to keep the system loosely coupled. I especially wanted the system to be able to:

• Consume any specialized features or functions that were developed or available from other platforms, like image processing and computer vision libraries, or document conversion functionalities.
• Interact with any database, e.g. MS SQL, MySQL, NoSQL (e.g. MongoDB), SQLite, Oracle.
• Easily replace underlying implementation of any feature or functionality being used in the engine or being provided as a service to other systems.
• Easily isolate, diagnose and fix problem in any part of the system.

So I built the architecture, its classes, objects, functions and functionalities, features, flow constructs, and the way they interact, in such a way that they came together to form the framework effortlessly, without getting entangled or convoluted or coupled with each other that could make it difficult to easily replace, redefine, isolate or diagnose any of them.

The loose coupling of the system and its benefits manifested in various areas. For example:

• A robust easily pluggable and configurable logging mechanism that helped keep the system sanity and issues diagnosis always a piece of cake.
• A multi tiered configuration system, registry based for server level, configuration file based at server-service level, and database based for whole distributed system wide service level, with runtime adoption of configuration changes.
• Ease of ability to test drive any part of the system in isolation in a console application for testing, fixing or improvement and ease of incorporation back into the system.
• A distributed system running on multiple servers from any locations, with threads and processes running on all or any of them, working in unison with a central database.
• An event driven, data driven, and chronology driven real time response mechanism.
• Ability to provide web services whether XML based SOAP or JSON or Restful, using either Web API or WCF technologies or legacy web service.
• Being able to easily modify any feature based on client input.

The Design Dilemma

Anyone can need and want a whole lot of things. But when it comes to software design, one cannot have all the design benefits one wishes all at the same time. It is often a question about which aspects are preferred over others. In fact often times a set of design choices means inability to incorporate another set of design needs.

Here are some of the desirable choices that were sacrificed due to other higher needed choices.

• Portability.

One of the biggest advantages of Java based systems, being able to run the program/code/system on any platform, is absent in this framework that is implemented on Microsoft Windows platform only. Although a good thing to have, portability in my context doesn’t provide much benefit to any stakeholders, especially when compared to what benefits are achieved by foregoing portability.

• Prebuilt layer of components and designing IDE

A large set of domain specific components for all kinds of specific programming tasks and features related to enterprise integration, providing ability to quickly orchestrate a workflow process without ever having to actually write any code. This is the kind of capability typically available in a complete product for building enterprise solutions.

My created framework is, by design, a work in progress as it always keeps on adding new features and functionalities once they are implemented while creating a specific solution for delivery. But it always requires to write the core code for any new feature if it has already not been implemented.

I made this choice to avoid the dependency on third party product for resolving any issues in a system built on it, avoid the regular upgrading and migration of all built use cases and processes to newer versions of the product. This consequently translates to saved costs and downtimes for the client. I also avoid creating such a layer in the framework for the same reason.

The amount of time and effort saved by using a layer to design the system is dwarfed when compared to the amount of time and effort required to maintain this layer, its robustness, efficiency, and to create any new feature in it, and even to compile and deliver any output out of it. Moreover it adds even more amount of time and effort spent in trying to debug or resolve issues created in anything built on this layer, compared to directly writing the core code of the new feature as a robust loosely coupled piece to serve the system. The biggest problem with the additional layer is the huge restriction in the capabilities of what can be created on top of that layer as compared to directly writing the code using any underlying robust libraries instead.

• Automation and Standardization of Maintenance and Deployment

In my opinion, this is probably the most desired and useful feature that should be part of the framework but isn’t as yet. Automation and standardization of things like codebase maintenance, forking, backup procedures, system deployment procedures hugely add to the maturity, robustness and maintainability of a product beyond dependence on individuals. At present a lot of these are being handled by ad-hoc arrangements and manual procedures which makes it susceptible to human error. The choice of foregoing this important aspect has been due to the limited amount of time and team members, and projects deliveries to clients being higher priority. But it stays the top most needed feature that I plan to add to this framework in the future.

Back to the importance of Context

I want to take the liberty here to stress how sensitive are software design considerations to changes in context. Just change a little bit of context here or there, and the design choices will and should change. For example, in my context if there was an abundance of allocated resources to build this framework, the automation and standardization of maintenance and deployment procedures would already be part of the framework. In another instance, if one or more big clients were invested in embracing an enterprise platform for all of their systems, we would be creating systems built on top of that enterprise platform product. Another example is if we were designing a front end or user interface, then using designing IDEs would be a much better choice than wanting to write the code for everything. Similarly it is self explanatory how other differences in ones context would lead one to make completely different design choices.

The results I achieved with my design choices for this framework

1. Clients always got their desired features for their unique needs.
2. Maximized success rate and profit margin for the company.
3. Providing production support required least effort and maximum profit.
4. Clients were always exceptionally happy with being able to add or change any feature at any time after delivery and deployment of their solution.
5. It was always very easy to impress and close a potential client with a demonstration of their needed features that we could quickly develop.
6. Being able to easily and quickly integrate with any components, systems, channels and end points of the client or by any third party or developed by other vendors, was the biggest killer capability behind the framework’s success.