Designing experiments for succeeding or failing in a turbulent world

How to design experiments that surface flaws in your mental maps about what the future holds? Succeeding in turbulent times requires the commitment to action while knowing that your map of the future is flawed. Running experiments is the single most important activity singling out what will work and what doesn't. Based on the knowledge gained through the experiments and your increased understanding you can redesign your mental map and increase the likelihood of succeeding.

There are two domains we need to understand better to succeed. That is to identify deal killers and knowing what we are betting on.

• Identifying deal killers. Which events will kill your idea or business?
• Knowing what we are betting on. Which unique things are we dependent on to succeed? What will drive customers to our offering? Not the inverse from deal killers.

Getting the cold reality of flawed mental maps enables us to spot weaknesses and opportunities we didn't know beforehand. Experiments lets us under control and with limited bets get a better picture of how our mental map of the future actually fits with reality. Here are five ways of running experiments:

• Partial experiment. Test one attribute at a time. Test parts of a deal killer or bet.
• Design holistic experiments. Test multiple variables and the interactions between them on a small scale. Could also be a test launch before a larger roll-out.
• Explicitly consider trade-offs. Cheap Fast or Certain. But not all three at the same time. Limited budget? Go for cheap on the expense of certainty. Need it fast? Put in more money to match the speed and certainty needed.
• Stage your experiments. Start small and try out the major killers or bets early. Once decided - go all in for that variable and execute the experiment. In between experiments there are room for discussions - during execution it is focus on results. After each experiment evaluate and decide on next step based on results. Don't hesitate to change your mental map - after all that's what the experiment is all about. Validating and finding more/better facts for the mental map of the future.
• Avoid experiment creep. Stick to what you decided during an experiment. Don't let findings during the experiment let you deviate from the set experimentation plan.If the results are not as expected - don't interpret them in the most positive way. Change the mental map instead.

How are you running your experiments and what value do you get out of them to clear the fog of the future?

More to read:

• Don Sull: The Upside of Turbulence
• MIT Sloan: The 4 Ways IT is Driving Innovation. How IT helps drive experiments.
• HBR: How to design smart business experiments?
• Dan Ariely: Predictably Irrational

Has Enterprise Architecture reached its Plateau of Productivity?

Enterprise Architects pride themselves with driving IT from a business perspective. Are they doing that or is it just another case of technology driven IT? This time with structure and control as underlying forces.

What shall Enterprise Architects do to really benefit their organization and add value?
There are two main drivers that you as a business leader have to balance between; Exploitation and Innovation.

• Exploitation. Exploit any given opportunity to the maximum. Be as efficient and operate with low cost. The IT delivery itself should of course be Lean. But more importantly it should enable other departments to be lean in their processes and operations. Embrace possibilities. Don't be too reliant on standards, especially IT standards. Let each business area decide what is best for them in terms of functionality, not in which system to use. 
• Innovation. Innovate within the IT delivery. How can we help the business being more efficient? Have IT drive Innovation, from idea generation to replication of new products, services and business models across geographies. Enable exploitation of new innovations through effective and efficient IT solutions. What should they look like? Well, be close to the business and listen, invest in agile IT support systems that are evolving over time. Calculate at least 25% of IT costs to be spent on new development, not just IT operations and roll-out.

The aim for an Enterprise Architect is to enable, guide and steer the IT delivery between these two extremes. Pushing Exploitation to be as efficient and extract as much profits out of any given opportunity. Driving Innovation to create new opportunities and being agile enabling new sources of revenue.

The most important things to get right are then:

• Interfaces. Let interfaces drive the enterprise integration, not standard systems. Service Bus is a technological solution to a business problem. Get the business solution right then the technology will solve itself. Having interfaces, common data models and master data drive integration will definitely drive integration across business domains in large enterprises.
• Collaboration. Share ideas, solutions and experiences across all silos. Collaborate physically and digitally. Use video, IM, YouTube, Twitter, etc to share and discuss across the entire enterprise. "If Xerox knew what Xerox knows" is a famous quotation from an organization inventing everything but not exploiting anything to their benefit.

Plateau of Productivity? Yes, unless we focus on interfaces and collaboration instead of steering and controlling Enterprise Architects have definitely reached the Plateau of Productivity. What do you think are the constraints for enterprises making more usage of Enterprise Architects to avoid the Success Trap of staying in the Exploitation domain and getting stuck?

Organizational rust and cholesterol

Organizational rust and cholesterol develops in every successful organization over time. In the book "In search for excellence" Peters and Waterman identified the 43 most successful companies in the US in 1980. Out of those 48 companies that were outperforming all other US companies only three would make the list today. What happened to the others?

It's an easy question to ask and much harder to answer. The background issues can quite easily be identified, the cure is much harder to prescribe.
A successful organization develops over time one or more of the following:

• Misuse of resources. The once successful departments get the majority of resources out of old habits.
• Misaligned strategy. What was once a great strategy and market position is today just stubborn actions.
• Leadership dissonance. Blame and backstabbing is spreading throughout the leadership team on different levels.
• Organizational immobility. Misalignment and inability within the different organizational units to move forward. It's one step here and one step there but never synchronized and at the right time.

How can this situation be fixed or even avoided?

In a recent HBR article on the theme "Change for Change's Sake" the authors argument that organizations should practice small and larger regular changes to practice the change organizational muscle. They propose a questionnaire that can easily help identify when it is time to initiate a small or larger change in order to avoid rust and cholesterol. 

An interesting thought is to measure the number of ongoing changes in an organization. How well are we at actually change? Do we have an ongoing readiness for change? What different types of changes do we have ready to apply if and when we need them?

What level of rust do you have in your organization and which changes of different size can you make use of?

How to manage several business models at once?

Facing stiff competition from new business models, regardless if it's new or incumbent organizations, is a challenge for any company. Customer loyalty is jeopardized, profits and cost structures are under at risk and careers can go overboard. Developing new business models is a challenge of its own, but how should my organization respond to the challenge of a disrupting business model in conjunction with our existing way of doing business?


Netflix is an excellent example of a disruptive business model. It works in two ways; first it operates over the internet avoiding fixed costs and local administration, secondly it uses the collective intelligence of its customers to make recommendations based on past rentals. Combined Netflix offers their customers simple and easy movie rentals with much higher viewing experience. How to combat that for an existing video rental business?
Should we:

• Focus on the segment that wants local video rentals taking the risk that they eventually will switch over to the Netflix model?
• Setting up a new business that competes head-on with Netflix taking the risk that we never will be able to do it better than Netflix?
• Creating a completely new business that disrupts Netflix and thus puts us ahead of Netflix taking the risk that our current business will cease to exist?

The text book guide is to either focus on a smaller segments that are not interested in the Netflix model (Porter) or setup a completely separate business with its own value-chain attacking the disruptor (Christensen). 

My take on this problem is to analyse the basic need that the current business model caters for. Can we continue serving these customers so well that they will never switch? If that is the case then change the existing business into clearly market that value and let the other customers leave for the disruptor. Then we have a healthy business that clearly provides a stable customer value that will continue to be there. If not doing this radical step quickly you risk loosing so much money and credibility that there is no business worth restructuring.

Then launch a series of separate businesses as experimentation. You don't know if the disruptor will eventually succeed or if it will create new opportunity and need for other business models. These experiments should both compete head-on and disrupt the disruptor. What's important is that each of these businesses operate separately from the others with its own culture and value-chain where it is necessary according to the business model (customer value, revenue model, cost structure, partnerships and profits). Cooperate with the rest of the organization realizing economy-of-scale where the business model does not mandate it. 

When facing disruptive business models one must be clear that the disruptor is starting small and moves fast. When the disruptor becomes large enough to threaten your existing business it is generally too late. The best prescription against this is to continuously run business experiments in small scale in different locations involving parts of the organization. Then you will try out market opportunities, find better cost structures and be ready for disruption yourself.

Inspiration for the post is from MIT Sloan Management Review article What to do against disruptive business models?

IT driven innovation

IT becomes a major driver behind increased speed and results from innovation. With manufacturing becoming a commodity and marketing increasingly hard and expensive many organizations are turning to innovation as the new revenue source. Innovation can create higher valued products from existing manufacturing capacity and simplifies marketing through improved experience.

IT greatly speeds up innovation and increases the effect in four areas:

• Measurement. How do you measure the current state of affairs?
• Experimentation. What is the value of this idea?
• Sharing. Who can benefit from my new knowledge?
• Replication. How to scale the new innovation throughout the organization?

These areas are important as separate activities and most organizations do one or several of them. They reinforce each other further magnifying the effect from each area. Improving measurements gives you better data easier and cheaper. Experimentation thrives on faster and cheaper measurements. Sharing those results further increases the value across the organization. Replicating successful business innovation enables faster scale ups and shorter time to market.

Measurements is radically improving the way we collect data and sort it. Nano data from click streams, emails, discussion boards, web trends, CRM, supply-chain, etc. Business intelligence is tapping into this area; needs to tap into the very bits and pieces of the nano data and make it available to the organization.

Experimentation through IT tools based on improved measurements. Measurement both to feed into the idea generation and to evaluate the effect from different ideas. Retailers can use IT to efficiently instruct different displays and then use improved nano measurements to follow up sales and profitability. New business models can use IT to track the consumption of goods and services through RFID tags on individual levels.

Sharing the results of experiments and the measures improves others to innovate. They don't have to invent the wheel again, can get the exact data measure setup, sharing ideas that for some reason failed (sometimes the most important aspect as failures seldom gets documented and less often shared).

Replication of innovations once they've been evaluated and proven is the key to successfully reap the benefits. The first three areas help organizations find and nurture new innovations. IT makes it possible to take that innovation and roll it out quickly and seamlessly. IT drives replication of business processes themselves. An example is how a bank is replicating a new loan process across all branches, a new customer return policy, displaying new food products in a retail store. Measurements enable the follow up of how well replicated the innovation is.

Collection of nano-level data is the foundation for IT driven innovation and replicating the innovation across the organization enables fast time to market.

Cognitive Surplus

What did you do before the web? How was it possible to find a restaurant in those days? During breaks - what did you discuss?

The world used to be disconnected and communication was one-way - from producer to consumer (unless it was from Aunt Annie - then it was totally wasted ;-)). We consumed what others were producing - clothes, food, houses, TV, radio, music, etc. Today we still consume what others produce - with the big difference of us being able to search for what we would like to consume - gone are the days of mass-market consumption. Using Chris Anderson's words - The Long Tail becomes the new Mass-market. We still consume - but not the same stuff. And that's why we have so few common things to discuss.
The even bigger difference that emerges is that we nowadays not only consume - we produce - and for free!

Instead of just passively consume what others produce - we now turn our time and energy into producing. The time previously spent on TV alone represent 1000h/year. That passive time is now slowly being turned into creativity - especially among younger people. Contributing to blogs, wiki's, Twitter, Linux, Facebook, etc are the new time-killer - and this time we fill it with thoughts, creativity and true dialogue. Spare-time previously spent on TV is now being redirected into creativity.

In the future each of us have some 1000h/year to spend on creativity. Shall we continue to watch TV and passively consume what other produce or start contributing ourselves? This extra spare time is what researchers call the Cognitive Surplus - the current passive time possible to turn into creativity. All Wikipedia entries represents some 100 million hours compared to 500 billion hours of TV in Europe and US - there is a large pool of time potential out there. Combined with our drive to do things that are engaging, interesting and because they're the right things to do. Spare-time, sharing technology and motivation of interest will enable us to become more creative - creating new topics to discuss during breaks (not because we have to - just because we want and we can).

Which Wikipedia article will you contribute to tonight?

Instead of TV why not read any of the following for more inspiration:

• Great spare-time discussion between Shirky and Pink on Wired
• Cognitive Surplus by Clay Shirky
• What motivates us: Drive by Daniel Pink

Watch your extreme customers

Your extreme customers are leading you into the future - explicitly - if you just care to watch.

The Chinese appliance manufacturer Haier uses their maintenance and repair guys to watch what customers are doing with the appliances. Where are they used, in what environment, who is using them and for what purpose. When a lot of washing machines got clogged with dirt they did not just assume the customers were stupid - they investigated the root cause. What did they find? That their customers were cleaning vegetables in them - can you clean clothes you should be able to clean vegetables seems to be the logic. Did they put big warning signs on the appliance - DON'T WASH VEGETABLES? Did they sue them or tell their customers it was their own fault?

No they changed the design to enable washing of vegetables as well as clothes. They watched what their customers were using their products for, listened to why they wanted to use it that way and then changed their behavior and designed a machine meeting the needs. Did customers notice what Haier did? Were they happy? The answer was loud and clear - market share in rural China went from single digits to 24% in 3 months in a market growing exponentially.

How do you start listening to your extreme customers?
First of all, in an article in Harvard Business Review April 2010 issue (Behold your extreme customers) researchers found that only 8% of managers cater to the needs of extreme consumers - so you will be pretty alone. Your extreme consumers can be engaged through for example:

• Let customers take ownership. Let the customers form communities, groups and influence/control product development. Lego took a whole new route after gathering their most devoted users and developed Lego NXT, the Robot Lego.
• Ask customers for help. Involve extreme customers in focus groups, ask them to test products to the extreme or suggest new designs.
• Sponsor events. Extreme events targeting the extreme customer. Costume dressing parties, all-you-can-eat festivals or invitation only parties where you must prove you're an extreme customer.
• Share consumers’ stories. Share in papers, advertising or on the web the most extreme usage of your products. Let your extreme customer become the hero.

How can you let your extreme customers drive your product development, marketing and brandbuilding?