Phasing your BI solution

Tips for successfully taking a phased approach to building out a Business Intelligence capability.

We are seeing more and more customers shy away from making a one-off big bang investment in large Information Management solutions. The trend is towards an approach that is more incremental in nature and focused on delivering Business Intelligence capability to the organisation in bite sized steps.

This phased approach has a lot going for it: quicker wins for the business, keeping the business engaged, staying relevant, better and quicker ROI. At the same time avoiding the issues with the big Information Management project that tended to be plagued by just the opposite: long delivery times, the business gets bored and disengaged, the business needs can change and hard to justify the investment.

A corollary is if you are going to fail – fail early. With an phased approach it will pretty quickly become apparent if the project won’t give the business what it needs; whereas the big IM project would often get a long way down the path before it was obvious it was off the rails and in the end if it delivered at all, at least one of the following success parameters would have been abandoned in order to deliver something: on-time, on-budget, to scope.

All of the above sounds like a convincing argument for the phased approach. But there are hidden risks and costs. Without a view of the bigger picture of the end game, both from a business and technical view, there is a risk that a series of small projects will deliver tactical point solutions that meet immediate needs. But can these point solutions be integrated into a bigger overarching solution? How much rework is required along the line to make previous deliverables work with later deliverables? Are they easy to enhance as business needs change?

From experience, there are a couple of tricks to achieving the best outcomes when using an phased approach to building out Business Intelligence capability.

Have a Road Map

The best outcomes are achieved when some effort is put into having a view of the end game and an initial idea of the steps required to incrementally get there. Think of it as a Road Map.

The key points it needs to convey:

• The ultimate objective in terms of both why and what.
• The steps to be taken: what each phase delivers, the business benefit and the contribution to the ultimate objective.
• The technology to be used. IT input to this is critical to assure alignment with its architecture and what can be supported.
• Data sources. These may be different for different phases. In fact, often the phasing can be dictated by adding different data sources to a growing body of data with each phase. Defining the data source in advance also helps forewarn IT of the data access requirements.
• How to approach master data.
• Any required short term work around. Sometimes the path to Nevada is not linear. For expedience, a short term work around may be needed until resolved in a later phase. This is a pragmatic, but acceptable approach; but best identified so that the rationale is understood when the work around is built.

This doesn't have to be a huge, expensive Information Management Strategy that sits on the shelf too big to read. It can be as light as a few pages, preferably more visual than wordy.

Once you have a Road Map use it and review it after each phase to make sure it is still aligned to the business objectives. Businesses change, the Road Map needs to change with it.

And a High Level Architecture

A high level, overarching Architecture also helps. This defines the architecture components, especially any shared ones, as well as the technologies to be used and any design principles. The aim is to ensure that each increment designs its components in a way that is consistent with the architecture and therefore consistent with other components. It also helps ensure that the different components can work together.

Setting down architecture principles can also help ensure the phases align to the overall goal. An example is dictating that all components should be built to support real-time reporting. For an individual phase this might not matter, but if an end goal is to ultimately have real-time reporting then each component should be built with this in mind.

Again, lighter is better: a few pages, a diagrammatic representation of the architecture and a few words to explain the components and any principles that need to be adhered to. It should be comprehensible to both the business and IT.

Master Data Management matters

Getting key reference and master data right is important. If each step has its own set of reference data, it becomes much more difficult to bring together the different deliverables into an overall solution. This often happens when each phase addresses a different source system when those systems don’t share the same reference data; in isolation the deliverables for each phase can deliver what was expected, but they won’t build up into a holistic set of business intelligence components that work together.

Determine the approach to be taken to master data as part of the Roadmap and then to implement that approach in each phase. Master data management (MDM) tools exist to manage both master and reference data. The technical side of implementing a MDM solution is fairly straight forward. The issue is that it also requires a commitment from the subject matter experts within the business who understand the data and the business, to help define the rules and potentially maintain the data within the MDM solution.

Getting master and reference data right will make a big difference to the overall usability of the analytical solution being built. The investment of time by the subject matter experts will be repaid many times over as they will ultimately be the ones benefiting from being able to use the analytical solution to do their analysis.

Waterfall v Agile - Or Hybrid

An incremental approach lends itself to agile: short phases (aka sprints), heavy business involvement and a focus on business outcomes. But there are aspects that may be better suited to using a more traditional waterfall approach. Agile works best where business users can see the product being developed and where the cost of rework (inherent in an agile approach) is less than the cost of rigorous analysis and design that is a feature of waterfall.

Moving data around, for example from an operational database to an analytics data mart, requires attention to detail, often involves complex logic to implement transformation rules and needs appropriate testing to validate. Rework is expensive. Arguably, these tasks are better performed using waterfall to minimise the cost of rework. However, once the data is loaded, agile can be used to build the business artefacts (reports, dashboards, advanced analytics routines).

Where the overall solution includes some form of data warehouse or data mart as well as the business user artefacts, it is worth considering a hybrid approach where the data handling is done under a waterfall approach while the business artefacts are developed using an agile approach.

Think Strategic Act Tactical

Using a phased approach to build a Business Intelligence solution doesn’t mean not thinking strategically. Having a big picture, knowing where you are going and how you will get there, will give the organisation a much better chance of success. Even though the phased approach is a rejection of the traditional big Information Management project, some aspects of the IM project are worth bringing forward, even if used only lightly.

Thinking strategically means adopting the best of both worlds to get the best outcome.