The Optimization Process

Testing and Optimization is should be an essential foundation of any marketing campaign, experience or customer journey. While it does take an initial investment in order to launch an optimization practice, the return can quickly be realized if you stick to this process.

The great thing about The Optimization Process is that it can be applied to many experiences. This includes not only digital experiences, but also for media, campaigns, contact centers, stand-alone applications and mobile apps.

At any given time, I suggest that 10-20% of your audience be under some type of test. In my experience, this number is a safe value to work with so that your performance will not suffer too much overall but you'll be able to grow upon your current communications. Without testing, you could be subject to customer fatigue and market stagnation.

The Optimization Process has six core steps that are essential to building a robust Optimization Program that is focused on iterative improvements and value-focused testing.

1. The first step of any optimization program starts with Ideation. This step can include not only the testing and/or analytics discipline but also other stakeholders that have an interest in improving the digital or customer experience. The end result of Ideation should be a list of testing and optimization ideas that can be evaluated in the next step.
2. As as result of Ideation, you should have a list of test ideas that various team members have come up with. It is important to Plan and Prioritize these test ideas. Often times, optimization practices will be forced to implement a tests that have the attention of higher-level stakeholders based on perceived test value. However, if possible I think it's important to implement a test based on it's calculated value using existing data. If a test is only expected to impact a small population, the results of the test would take longer to determine and the impact would be smaller compared to a test that could be exposed to a larger universe. In addition, it's important to consider the cost-benefit of implementing a test. If a test takes more effort to implement, that costs should be calculated compared to easier tests against the same audience.
3. Once you have selected an experiment to run, you will want to determine the Test Design. In determining the design of the Test, you will decide where exactly it will run, against what audiences and also determine what metrics will be used to decide the winning experience. Exposing a test to the wrong audience or improper setup could result in invalid test results.
4. Following test design, the next step is Implementation & Execution is the act of setting up the test and receiving live feedback. The implementation step is critical to make sure that the test is aligned to the original Test Design. The details of the implementation will depend on the channel and test that is to be executed. There is also a chance that the test will be sent back to get more feedback if the test itself can not be implemented according to the design. It is also important to QA the test before pushing live to make sure it doesn't impact anything it shouldn't and operates as desired. If the test can be implemented correctly, the test should be executed or 'pushed live'. When the experiment is live, be sure to evaluate the incoming data to make sure the results are as expected.
5. After the test is complete, it is essential to perform Analysis. This analysis is performed to interpret the results and tell a story whether the best performing option was the challenger or even the control. The analysis can also tell if subsequent tests should be considered for more learning and optimization. These subsequent tests are an essential component of iterative testing. By continuously testing, you are making sure that the experience you are providing to your customers is the one that best suites their needs.
6. The final step in The Optimization Process and one that is normally forgotten is Activation. This step is taking the results of the experiment and rolling them out for all desired audiences. It's common to run a test and to never roll out the results to a wider audience. By not doing this, you are defeating the whole purpose of the test. If the desired outcome of a test is to increase performance and you don't roll that out to a larger audience, you aren't capitalizing on those learnings.

Of course this process is an outline of a framework and you may have to adjust at times for your organization and optimization program. However, in my experience, this process can be applied in many organizations and covers all the bases of essential steps for a proper optimization program.

If you have feel further questions or comments, feel free to message or comment. Feedback is welcome.