Welcome to the blogosphere Gary! Good to see you here.

I have a question that relates to feedback and expertise, and it's linked to something that has been bothering me for a while now, and that I've been discussing with colleagues at KM and Information management conferences. The real question on my mind is, how do knowledge managers become competent and develop expertise in what they do? And that's linked to the question of whether a true "profession" of knowledge management can develop.

What we see is two problems with feedback: first is that the cycle time between inputs and activity and discernible results and outcomes is very long in KM initiatives. Often it's really hard to discern the linkages between effort and results. The second problem is that knowledge managers often move jobs mid way through a cycle so getting complete exposure to a true or "full" feedback loop is hard. So it's relatively rare for knowledge managers inside organisations to get the ability to bed down multiple patterns of experience of a type of activity (in fact, that's one of the main reasons I decided to become a consultant), and often they get partial fragments of experience.

I wondered if you had studied the acquisition of expertise in such circumstances, where feedback loops are very long, or are fragmented?

If I may, I'll add a third "problem with feedback" to Patrick's comment: the rarity of effective "negative" feedback, i.e. of admissions of failure (or even of "incomplete success"). This is, of course, not unique to KM initiatives. A "testable hypothesis" is a moot point if we cannot readily observe whether the hypothesis has failed.

Patrick and Keith …

You both raise a critical issue about how confusions and inadequacies of feedback make it hard to develop expertise. I completely agree. In fact, I have written about this problem and explained that simplistic ideas about feedback aren’t adequate. In most settings, feedback requires sensemaking because of the time lags and the difficulty of sorting out intervening events to figure out cause-effect relationships. So I don’t believe the KM community is unique here.

I don’t think that these problems prevent people in the KM community from developing expertise but they slow the process down and reduce the level that you can achieve (say, compared to chess grandmasters). The issue of standards for expertise is a whole other topic, though, and perhaps I’ll blog about it later during my guest appearance on this site.

It seems to me that the KM community is perfectly situated to deal with this problem by creating a repository of incident accounts – stories – describing projects and their outcomes. This could overcome the second problem you raise (turnover) along with the first problem (time lags). It wouldn’t eliminate the problem Keith identifies, unwillingness to describe failures. But we all know that failures can be the most informative incidents. By masking the identity of the sponsor and tinkering with other identifying details, you may be able to accumulate some of these as well. I suspect that professionals may be highly motivated to write about failures and to speculate about their causes.

Gary

Hi Gary,
Very interesting post (of course).
There is perhaps a range in the extent to which stories and beliefs match reality; in High Reliability Organisations, there is quite a good match (though some interesting technical mismatches, and a poorer match at a command/management level). There are some domains where the match is hard to establish, but where people's estimate of the match is way out; the interest in Evidence Based Medicine is an attempt to improve the match for doctors. The finance field is known to have a very low match, but being able to tell a good story (confabulate?) can be important to personal financial growth/survival. The 'Black Swan' tells good stories about the inability to tell stories in this sphere.

So, in addition to feedback, some sort of estimate of likely success is important (used to be called humility I believe).

As regards KM,
http://www.socialtext.net/cases2/index.cgi?cases_2_0
might be the sort of repository you mean?
regards
brian

Gary,
I am, as always, impressed by the depth of your writings. (I was thinking of putting "Long time listener, first time caller" on this, but I sometimes haunt the CE blog, so I'm not a total newbie.) My question is - What is expertise? From context it seems you are referring to 1) The ability to accurately predict the future, and 2) The ability to execute within that predicted context in order to impact the outcome in a manner that is aligned with your desired results.

The first is the reason why people in the stock market are not, and can never be, experts. The second is what really generates the challenge. It may be easy to see the a disaster is looming, but what can be done to avoid / mitigate? KM, as a huge, amorphous blob, cannot as a whole be expected to assist much in either of these tasks.

But as you mention, failures often teach more than successes, but people really don't like to share failures. There is some hope that baby steps can be taken. I have seen programs in the US government that are starting to attempt to address failures as well as successes. Given the billions I see being thrown away on Lessons Not Learned programs, etc. baby steps are about all we can sometimes hope for. But there does seem to be a little progress.
Wayne Zandbergen

Many thanks for the paper Gary it is really helpful. A couple of follow-up observations:

I like the pinpointing of mental models and sensemaking as a strategy. Unlike the scenario in your paper, however, there are no real "experts" available to be instructors - by which I mean people who can diagnose faulty mental models and use deliberate strategies to break them down and stimulate new ones. We're all pretty primitive in KM right now, and our mental models are not far above magic.

I'm not sure that the "failure" stories route has much promise right now. In principle it should, but we're often not in a position to make clear distinctions between success and failure. And we don't like to tell stories about our failures to a public audience. I think stories about "challenges" are probably easier to elicit, and they should be interesting from a learning point of view because a challenge is where we become aware of the limits of our ability. Yet even here, we're far better sharing the stories face to face with people we know than sharing in public. Case in point: the social software cases referenced by Wayne above: most of the 16 cases put "N/a" in the "Hurdles/Challenges" section of the template, one case mistakes it for a "Benefits" section, one case gives very generic principles, and one case lists implementation challenges, mostly technical.

So the collective sensemaking approach where we pass multiple sets of eyes and perceptions over our shared experiences seems to me the way to go but it will probably only work well in small face to face groups. You talk in your paper about decision games as a strategy, and we've used this with some success as a group exercise. Some years ago we interviewed 20 or so knowledge managers confidentially about the challenges they wee facing in their work, and built the insights into five fictional decision games for starting-out nowledge managers. They work well in small group settings. There's also a technique called "play of life" which is a table top exercise where one member brings a significant challenge to the group, and represents it on the tabletop using figurines. The group then spends 20 minutes or so just asking questions about it. Then they can play the scenario forwards or backwards to look at alternatives and explore options. I think it would work well fo the sensemaking you describe.

But I guess the real problem (for an evolving profession) is how to scale this so that it becomes pervasive and ritualised as a collective approach?