Qualitative research privileges the expert or experts in being able to analyze the content of data collected from interviews, focus groups, and other sources. This effort in itself is subject to cognitive bias as the researcher will see things in the content of such materials through the ‘lens of their expertise’. Another limitation is the fact that often the content is incomplete in terms of full context and it is likely to be ambiguous. Such content will then draw on the expert to ‘fill in the gaps’ with their interpretation which overlays meaning based on their expertise and judgement. The third limitation of typical approaches to qualitative research is the challenge high volumes of unstructured data poses. How does a researcher effectively and rapidly study high volumes of unstructured data in a comprehensive way without getting locked into their own biases?

The approach advocated by Cognitive Edge involves engaging the contributor of content in the analysis. In this effort we work towards distributing the cognitive load of analysis to a large population rather than concentrating the cognitive load on a small research team. The way this works is that a research team constructs a wide-scoping signification framework (see a new signification element, the triad, in one of Dave’s blogs) that is purposefully setup to reveal deep insights related to the domain of inquiry or objectives of a project. So for example if you had a project that had the general public contribute personal stories of their families experiences with the current economic crisis, you as a researcher might be interested in understanding the overall populations degree of hope, despair, fear, trust in the financial system, and possibly some other dimensions. After a respondent shares a story they then answer a series of specially designed signification questions (i.e. related in this example to hope, despair, fear, trust) about their story which not only interprets the content but adds layers of meaning that is not in the content itself. In this sense the contributor of the qualitative data contributes additional meaning to the content at the time of entry. As these interpretations aggregate across 100s if not 1000s of entries, emergent effects can be identified across large volumes of data using large volume data visualizations. In essence the meta-data (the question responses regarding the stories) provide insights into the collective perspective as defined by the general domain of inquiry - the example given being the collective experiences of families living through the financial crisis. The researcher now does not need to process 1000s of entries of unstructured data themselves, instead they leverage the meta-data contributed by the content contributor and dive into the content on an exception basis. This latter point is critical in minimizing the skewed interpretive effects of cognitive bias in the researcher. If a researcher is forced to move between visual patterns in meta data and then into content related to such patterns (or lack of patterns) then they have a better chance of disrupting pattern entrainment related to their cognitive bias. In addition to the visualization of patterns in the interpretive data we can also leverage statistics to identify trends and other patterns and rely on the content behind such statistics to add meaning to the numbers. In this sense the approach brings together both qualitative and quantitative aspects of research.

So back to the title of this post - “How do you make sense of 2000 ideas?” I summarize a recent project I worked on. In January an organization launched into an ambitious project of consulting across an employee base of approximately 4000 staff for ideas of ways to transform their business. The intent was to get all sorts of ideas from various contexts of the work environment from nearly every corner of the organization. The time to gather analyze and complete a final report on the effort was a mere 6 weeks. In less than 3 weeks we had over 2000 well thought out and articulated ideas contributed by over 750 respondents (nearly a 19% response rate). In the three weeks following that the meta-data was reviewed by a team of three project leads with the support of me using SenseMakerTM Explorer and other supplementary tools. A report was finalized and presented to the leadership team of this organization. In comparison to a similar effort by another part of the organization which about a year earlier gathered around 500 ideas from staff, their analysis effort took in excess of 6 months and was incredibly difficult since the interpretation of the raw data was put entirely onto a small team of researchers. Compared with 2000 ideas which were interpreted by over 750 individuals the distribution of the cognitive load was highly effective - not to mention that the contributors of the ideas have a far deeper sense of the meaning of what they wrote compared with what a researcher can make sense of only looking at their content and in most cases completely disconnected from their context.

So when you consider tapping into your employees, customers, or citizens on a mass consultation process don’t put the entire interpretive load on a small research team no matter how good they are at what they do. Distribute the cognitive load to the contributors in a way that gets deeper insights from the context of the content in the submissions. A process of inquiry that integrates engagement into the process will provide you as a leader the ability to quickly move forward and leverage your organizations readiness and willingness to transform, all according to a series of options they have qualified and analyzed for you. Oh and by the way the diversity and large range of perspectives that emerge in such data will force you to think differently and innovate -- so be prepared!

PS - the image in this post is from the 2000 idea data set (anonymized). The scatter plots are the interpretations of the ideas by the contributors themselves. Can you see the cluster emerging of ideas that are straight-forward and can be implemented immediately? (Hint: Graph in Row 1 Column 2)

It has been a long time since I have blogged and for some reason it has taken a while for me to jump into my own CE blog. With the transition to 2009, and since the end of a much needed holiday break, I have been reflecting on 2008 and the significant changes that have taken place. The first half of 2008 was an intense period of business activity along with the completion of building our new home of which a winter photo is shown. Now what made this a challenge was that my wife and I were managing the project ourselves (with the incredible support, help, and guidance of both of our fathers) and had to rely on timely availability of the necessary contractors. An intense period none the less. Following a pressured completion by mid-August (well at least complete enough to move in) we made our transition into our new home for the final weeks of the summer. Now the plan was suppose to be finish the house by the end of July, enjoy a couple weeks of summer vacation with the family early in August and then get back to work quickly organizing my new home office and into projects. What really happened is the vacation was consumed with house completion tasks and very minimal family enjoyment time was available during a working (albeit as a tradesperson) holiday. This resulted in starting work with boxes piled up around me in a spacious and nice new office feeling tired and quite exhausted. Not the best end to an exciting project with high anticipation of completion at the start.

Now the transition to the new home was marked with another new start and that being my role at Cognitive Edge. Although I was working closely with Steve and Dave in the months leading up to September 1, this marked the formal move and the start of my role at CE. Needless to say the intensity of balancing NuOptiks Consulting business with the construction of the new home for the months earlier was now consumed with an intense work period of getting moving on CE business. Fortunately the prior close working relationship made the transition easy from a relationship standpoint however becoming part of a larger team with a presence and activity around the world added a new dynamic to my work. It continues to be very enjoyable and intense and I hope to share how this journey evolves from my unique perspective.

New beginnings in multiple new homes. A nice way to finish 2008 and start 2009 with exciting momentum.