When does evidence based policy become policy based evidence? These days it would appear to be when a hypothesis held for ideological reasons needs to be proven. There are many ways to do this which are a little less blatant than the cartoon but have the same effect. One of the most obvious, given the way research is carried out these days, is to restrict the literature search terms to make sure than only favourable material is created. Far too much government policy is based on restricted summaries of selected papers, or hypothesis based interviews that are far from open. Few people really question these days, or fact check, or think about what is being presented and the assumptions behind it.
It can be worse a team of researchers identified what they called the paradox of unanimity. In a police line up they identified that as the number of independent observers agreeing on identification increased the probably of them being wrong decreased (Lachlan J. Gunn & others The Proceedings of The Royal Society A forthcoming). In the recent Volkswagen scandal it was the consistency of the results which alerted to the possibility of fraud.
Some time ago I agreed that we should change Jury trials from twelve people in a room, to four groups of three working independently of each other. That lead someone to remind my that in some Jewish Courts it use to be the case that unanimity meant you were judged innocent as it evidenced group think. I’m working on a service we can launch for corporate decision making based on that jury idea by the way. It will build on what we already do with MassSense, where we look at dominant descriptive patterns as well as outliers to provide confidence levels and also mitigation strategies. A lot more on that shortly and MassSense will be available for ad hoc use shortly for those who want to get in on the ground floor. The wider decision support project will be a CfAC membership one and that will be announced shortly as well.
I’m going to work some of this new thinking into the Cynefin and SenseMaking course on day three, to go with the work on decision mapping and organisational change. So if you want to be there for the early thinking book now!