Measuring the digitization

Nowadays, one of the biggest problems of the digital transformation is to recognize the gain obtained by the digital transformation itself.

It happens very often reading some myths and ways of applying around the topic; just as a matter of example, I would mention two different opinions: “the digital requires radical disruption of the value proposition” (Harvard Business review – Nathan Furr ); “digital transformation requires a shift in mindset, not just technology. Digitization is transforming business models, processes and strategies, and with all changes, what it requires is a change in mindset”( Daisy Chittilapilly, MD, Digital Transformation Office (DTO) for Cisco India and SAARC).

The digitization has many facets including cloud adoption (SaaS, PaaS, IaaS), use of artificial intelligence, use of advanced user interface technologies for better customer experience. It may also encompass the automation of business processes for improved efficiency. The IoT world we are living gives another dimension for the organizations in the direction of the digital (r)evolution.

Unceasingly the IT departments and the IT companies continue in bring technologies out, in proposing evolutions, revolutions, changing in mindsets; unfortunately many of those changes require big investments:

in the end, all those efforts have the objective of using digital tools to better serve the known customer need.

All this said, are the organizations 100% sure they are measuring the gain on the user efficiency or customer happiness: “as organizations launch digital transformation initiative, many discover they lack visibility into the actual end user experience.” (Ajay Gandhi, vice president of product marketing for Dynatrace).

Furthermore, if we want to proceed in the direction of measuring, are the engineers sure they have all the elements to establish clear KPIs, to answer some business questions like “if I invest 1 € in the digital transformation which gain I will have? In which terms? How I can measure the gain? What should I observe to demonstrate the percentage of increased efficiency, or the impact on the customer need?

Browsing and reading here and there, trying to investigate those KPIs, I got into the Apdex index.

Apdex was developed by an alliance of companies (2004) as an open standard to report, benchmark, and track application performance. It serves as an easy indication method to translate application performance benchmarks to a simple comparison metric measuring responsiveness over a period of time.

The apdex index is based on three application responsiveness zones:

Satisfied: The user is fully productive. This represents the time value (t seconds) below which users are not impeded by application response time.

Tolerating: The user notices performance lagging within responses greater than the threshold (Apdex t), but continues the process.

Frustrated: Performance with a response time greater than F seconds is unacceptable, and users may abandon the process.

Traditional use of the Apdex formula when applied to application response times as seen by the user:

Apdex = (Satisfactory samples + 0.5xTolerating samples + 0xFrustrated samples)/Total samples

Despite the simplicity of the formula the Apdex has not been so widely used, also for some sad events ( Chris Loosley, one of the Apdex most active promoters, died in 2012)

The Apdex scoring system was invented to introduce standardized scoring into the world of performance metrics. The online experience is in a continuous state of change, with more and more variables defining the user experience. Unfortunately, Apdex hasn’t evolved at the same pace.

Apdex scores are becoming more and more inaccurate due to their inability to attribute UX issues or success to a specific action, tool or solution. It simply doesn’t lead to any actionable insights. This is further exacerbated when significant parts of the code that renders the page is being delivered from off-page sources like code libraries or via third party integrations. (Ohad Hagai - SVP Marketing at Namogoo)

Continuing the analysis I found the concept of quality of experience (QoE): Quality of Experience (QoE) (wikipedia) is a measure of the delight or annoyance of a customer's experiences with a service (e.g., web browsing, phone call, TV broadcast). QoE focuses on the entire service experience; it is a holistic concept, similar to the field of user experience, but with its roots in telecommunication. QoE is an emerging multidisciplinary field based on social psychology, cognitive science, economics, and engineering science, focused on understanding overall human quality requirements: therefore nothing really measurable through a synthetic KPI.

Therefore if from a side we are observing the business teams more and more interested in innovating and following the technical roadmaps, on the flip side the IT teams should continue in working to provide, test, monitor and analyze key performance indicators to answer if I invest 1€ in the digital transformation which gain I will have.

In the next articles, I would like in continue studying the topic, trying to give my perspective in achieving these answers.