« Silos & sinecures | Main | Oh please .... »

Power laws & abductive research

Screen shot 2010-08-08 at 14.54.43.png I promised a more substantive post about our Friday session at the AoM on new forms of research, and I want to do it by summarising aspects of Max Boisot's final presentation, which did an excellent point of putting everything in context. He neatly summed up the three types of inference in this picture.

  • Deduction (show as the small dark dots) which delivers certainty and is based on logic.
  • Induction (blue areas) which delivers probability and is based on repeatable events
  • Abduction (the wider turquoise area) which delivers plausibility and is based on coherence with prior experience.

He rightly headed the slide Complements, not alternatives. Abductive techniques (and SenseMaker® is an example of a tool to support such an approach) are in effect means of generating coherent hypotheses under conditions of uncertainty. The coherence word is key here, just because we don't know everything it doesn't mean that all ideas have equal value. Creationism for example remains incoherent to the facts, while evolution is coherent, even if it doesn't account for everything and still contains errors. In complexity based strategy coherence is a key test for each safe-fail experiment.

Screen shot 2010-08-08 at 15.18.01.pngBefore he reached that point Max had built a highly coherent narrative around the idea of dots and linkages. I've shown the whole sequence on the right. Dots here are events, and the way we join things up is by narrative. We make sense of the world by explaining connections between things, the patterns that form from the dots and the linkages. Its simplified here of course with only four dots, as the dots increase the number of possible linkages increase. With four dots there are six linkages and sixty four possible patterns, if we go up to 10 dots then there are 45 links and 35,184,370,000,000 patterns. But we will come back to volume issue later.

In the second slide we see the difference between deduction and induction. With deduction there is only possible answer, it can be computed. With induction we can predict a range of events based on what we already see. The trouble is that the world is not always this simple. In practice (slide 3) we have inconclusive patterns and its all too easy to ignore events and linkages that do't fit the patterns of our expectation.

Abduction on the other hand (slide 4) is all about choosing between different narratives, or allowing competing narratives to run in practice to test for coherence, another way of saying safe-fail experimentation.

Now this leads us on to the integration of this with the whole power law concept. Most risk is assessed on the basis of a normal or Gaussian world in which means make sense and events that fall beyond a certain number of standard deviations can be ignored. However in nature if we plot a log scale of frequency against size then we get a power law as shown in the picture below. Also known as a Pareto distribution here was have to account for the fact that events which are outliers in a Gaussian world tend to have a higher probability given the fat tail of a Pareto distribution, we can't afford to ignore them, but we are dealing with small sample sizes which reduce the value of induction.

Max's integrative slide shows this progression from the world of Gassian distributions that dominate business school thinking, to the more uncertain Paretian world in which we are dealing with higher levels of uncertainty.

Max's conclusions (with some paraphrasing on my part) were:

  • We have to match the inferential strategy we adopt to the level of uncertainty we face.
  • Uncertainty is the surface manifestation of complexity at work
  • Abduction is the most tentative of the three inferential strategies
  • It allows anticipation but not prediction
  • In an increasingly complex world we increasingly have to settle for anticipation

Screen shot 2010-08-08 at 21.34.43.pngNow with SenseMaker® we argue that we can reduce that uncertainty by volume of material and self-signification of that material, i.e. we can make abduction less tentative, but you don't have to accept that argument to get the basic point. The statistical tools currently used in business are inadequate to the levels of uncertainty and complexity that we face.

Comments (1)

Uncertainty _may be_ the surface manifestation of complexity at work. It can arise any time cause and effect are not well-understood by the observers. In these cases the observer might be able to consult a relevant expert, and coherence could be applied in qualifying experts.

Given society's reliance on experts, I wonder whether you've described practical techniques (e.g. applied abduction) to separate experts from charlatans.

Cheers,
-Eli.

Post a comment




Remember Me?

(you may use HTML tags for style)