Machine Learning Development: Scribble - Squiggle - Straight
“Machine Learning (ML) development is innovation”. For me, it comes down to the definition of “innovate”: “to make changes in something established, especially by introducing new methods, ideas, or products”. Sure, the capability to develop ML is at least tested in most businesses. But inherently, we are building ML outputs designed to introduce a new method of conducting a business process.
As such, an innovation mindset is required from the launch of a ML project. How you structure development is different for innovation than for something more well-founded (such as a website; although a fair argument could be made for this being Innovation also). Some of the hardest questions you'll have to answer along the way are:
I've started to use the attached plot to describe this process and the required Lean mindset to find the ML solution “in the dark”: we're moving on a journey from Scribble to Squiggle to Straight. The goals at each of these phases requires a different mindset.
Scribble: early onset of the project. We know that there's a high level problem which has been flagged by one or more people and that we want to prioritize discovery. We can run some solution design workshops to get a better understanding of the business problem and to get an early read on the Feasibility, Viability and Desirability of potential solutions, but our understanding is hypothetical and the generation of results could identify where some of our initial assumptions are wrong. It's important in this stage to shorten the Build-Measure-Learn iterations so we are getting this invaluable feedback early and often, to make informed decisions to Persevere, Pivot or Fail a development thread. (Side note: this is the step which is often ignored in favor of “requirements gathering”. The risk with the framing of defining requirements without real feedback is that it assumes that what we hypothesize about at the start of a project is true, without the rigor of testing).
Recommended by LinkedIn
Squiggle: this is the phase of the project where you’ve settled on a hypothetical solution which meets the standards of hypothetical Feasibility (can we build it?), Viability (will it achieve ROI?) and Desirability (do people want to use/demonstrated how they can use it?). We have been able to have some firmer requirements formed (based off of rigorous testing rather than hypotheticals) and can now “fine-tune” the solution. The need for constant feedback is less here and you may be able to extend your development cycles, as we’ve been able to reduce the risk by demonstrating results over small tests for each of the requirements. Value is still hypothetical until we can deploy the production solution, which itself should be taking shape at this phase.
Straight: We have a solution ready to be productionized. The most defining feature of this phase is that the uncertainty has been removed. We know which direction development is heading and we can head there in well-defined, well-founded “straight lines”. Note: “productionized” does not mean the same for all ML outputs, and what the desired output to alter the business process would have been tested in Scribble and measured in Squiggle. This step is not without risk, however, as there is a high-risk “gated” step into production; we’re moving from Innovation risk into Disruptive risk. We also face the challenges of having the right platform and capability in place at this phase. Model deployment, maintenance and monitoring can all fall under this phase. We are now realising value, rather than building toward hypothetical value.
If you’d like to hear more about how Lean methodology and Design Thinking can be applied to get the most out of ML development, please give me a follow and don’t hesitate to reach out.
(image courtesy of Strategyzer)
