Clearing Up the Confusion: How Business Experiments Are Shedding Light in a World of Decision Uncertainty.
The world in which we live is uncertain. While it is true that predicting the future is impossible, we can calculate with some degree of confidence how much risk is involved when choosing one decision over another. The difference between knowing the risks and being uncertain is invaluable for all decision-makers needing to adapt in a constantly changing and competitive market. In efforts to reduce this uncertainty, and instead provide measurable probabilities of the risks involved in decision-making, a novel and scientifically grounded method to inform business decisions is fast becoming the preferred choice amongst companies. These companies see the value in testing their new ideas rigorously before implementing them on a large scale. This is the business experiment.
What are business experiments?
Intuition is vital, but it may not always be a reliable predictor of future success. In contrast, business experiments are specifically designed to test new business models using strategic interventions to produce empirically reliable results. By experimenting before implementing new business models, the risk of failure can be measured, giving decision-makers the confidence they need to select the model that is the most effective. Some of the benefits of shifting towards a small-scale business experiment strategy include: avoiding embarrassing overhauls, redirecting or improving on existing strategies, testing between two or more ideas, or simply developing more usable insights that can be scaled across your organisation and inform and direct your understanding of your clients, products and service offerings in future.
The power and value of business experimentation for supporting decision-making is its application of rigorous, scientific methodology. At their core, they are built upon the foundations of seventy years of clinical research. When it comes to testing the efficacy of a new medication, rigorous planning of each part of the experimental design is taken very seriously. Careful planning to achieve results that are valid and reliable is of the utmost importance when the costs of making mistakes are high. Why should business decisions be any different? Running experiments that are well-planned and scientifically valid provides valuable insight that can be used to re-design, re-evaluate and make informed changes to products, strategy or policy without expending extra effort and expense. And, most especially, without guessing.
When expert intuition fails
Ron Johnson left Apple in 2011 after a highly successful career as the Senior Vice President of Retail Operations, to become the Chief Executive Officer of J.C Penney, an American department store. Not only did Johnsons’ expertise and leadership contribute to Apples’ positive and ‘hip’ image, but he was also hailed as the pioneer of the Apple Retail Stores and the Genius Bar.
On assuming the position at J.C Penney, Johnson made a public announcement that he would be making large changes to the company’s pricing strategy. At the time, 50–70% of all sales were at discounted prices. Johnson proclaimed these to be “fake prices” which would be discontinued to make way for a simpler pricing scheme. The new strategy saw J.C Penney adopt an ‘everyday low pricing’ scheme instead of their traditional offering of discounts and sales coupons. Under this new strategy, price tags were also to be rounded up to the nearest dollar, eliminating prices previously ending in .99.
However, the new strategy backfired and within a year sales fell by 25%, resulting in a net loss of $985 million. By 2013 J.C Penney’s share value dropped by 51% and Johnson was ousted only 17 months after he began. In an interview with Business Week after the failure of his strategy he stated, “I thought people were just tired of coupons and all this stuff”. He went on to state that, “The reality is all of the couponing we did, there were a certain part of the customers that loved that. They gravitated to stores that competed that way. So our core customer, I think, was much more dependent and enjoyed coupons more than I understood”.
Johnson made two mistakes; he misread the customer and he did not test his ideas in advance. He believed — as many others do — that his experience and intuition was the best indicator of how to proceed with developing a new strategy. He failed to realise that J.C Penney’s customers — primarily lower to middle-class families — were accustomed to shopping for discounts. Furthermore, he missed a key component of the discount shopping experience which is that purchasing an item at a discounted price is more enjoyable and satisfying than saving the equivalent (and largely invisible) amount on each item. Though we cannot be certain of the results, had Johnson tested his new strategy in advance he could have benefitted from a better understanding of the customer and abandoned (or at least adjusted) the new pricing structures, saving the company millions as well as his job.
Sweeping the nation
Aisha Pandor and Alen Ribic began the South African startup SweepSouth in 2014 after the difficulty they faced finding a temporary domestic cleaner over the December holiday period. Their unconventional idea was to create a company that would allow home owners to request a temporary cleaner online and at short notice. They developed a website and online application form that allows home owners to fill in their contact details and preferences such as; their mode of payment and whether cleaning products are required. Shortly afterwards, home owners receives the domestic cleaner’s information. Once the process has been confirmed, the domestic cleaner arrives at the prescribed date and time, cleans and then leaves.
As a technology-driven platform, Aisha and Alen needed to consistently refine their process in order to cater to their customers as well as the domestic cleaners they partner with. A variety of barriers presented themselves early on, and a strategy to uncover their causes and break them down was essential to building their client base and attracting investors. They began by refining their product and webpage, using a process of running three to ten experiments regularly using A/B testing. With clear goals in mind, the results of their experiments provided the ‘low hanging fruit’ that the entrepreneurs needed to develop a user-friendly interface and a simple yet effective product.
SweepSouth was founded in Cape Town but has since expanded to Johannesburg, Pretoria and Durban. Furthermore, the entrepreneurial couple received additional funding of R10 million in 2016. One such hypothesis that they tested was where to place the sign-up form when customers visit their webpage. They questioned whether it would be most effective at the top of the webpage, as a first contact, or at the bottom requiring the customer to scroll down through the content before entering their details into the system. Other companies such as Google, Twitter, Amazon and eBay use the same constant data gathering process from small scale experiments to refine their processes and stay ahead of current trends. Looking at the example of SweepSouth, experimentation shows itself to be beneficial not only to large multinationals like Google, Amazon and eBay but also to startups.
Even negative feedback is positive feedback
Using experiments, decisions can be based on facts rather than ‘gut feelings’ and assumptions. The insights gained by evaluating the results of the experiments can be fed back into the business to adjust new programmes and marketing strategies, better align targets, increase profits and improve user experience. Starting with small goal-driven targets and then actively collecting more variables to answer ever-growing business questions helps to refine strategic changes prior to the full-scale implementation of a new strategy. Making mistakes is an important part of idea generation, but experiments help to avoid situations like that of J.C Penney by encouraging an atmosphere of pre-emptive and controlled trial and error. Making mistakes in a controlled environment and on a small scale is the key to minimising any risks related to implementing strategies when the results are uncertain and the costs are high.
American convenience store chain, Wawa, wanted to add a flatbread breakfast — which had previously received positive feedback in spot tests — as a fixed menu item. After performing rigorous experiments, however, they found that by adding this option they would lose profits from the other options on the menu. The idea was killed immediately before it could be implemented. The process of experimentation may reject many more ideas than it ends up generating, but it is worth the time and money. What is gained is the confidence that a winning strategy will eventually emerge. Running ongoing experiments allows a company to gain knowledge that can inform their future decisions by providing constant feedback on what works and what doesn’t.
Rather be safe than sorry
The collection and diversity of insightful data gleaned from business experiments helps companies adapt to an ever-changing and uncertain market environment. Innovative changes are not something that should be left strictly up to intuition because humans make mistakes. Decision-makers can be led down the wrong path by misreading customers and allowing over-confidence to bias their decisions. Often uninformed ideas — that at first excite and look as if they will achieve positive change — can end up being costly failures.
Experiments challenge conventional wisdom through rigorous methodology that provides the confidence to make smart business decisions. Two facts are clear: it is better to be safe than sorry and it is vital to stay ahead of competitors while encouraging growth. Invaluable insight is to be gained at every available opportunity.
