Now I should declare a commercial interest here.  I've been working in this field for the best part of two decades, and given the opportunity to develop software that came along  four years ago it would have been hypocritical not to incorporate my ideas into that development.  So some of the things I am going to talk about are a feature of the design of SenseMaker™, others are methods which are a part of the open source aspects of Cognitive Edge, others are just ideas.  I am currently working on a draft white paper on horizon scanning which will be ready for KM World, but if anyone wants an advance copy drop me a note (you get it in return for reviewing the content).  I've also blogged on most of these items over the past year or so in more detail; a few key word searches may find some more material.

Now I don't claim that this is a complete list, but its a good one to start from.  Each of these subjects deserves a book chapter in its own right, so accept the fact that a two paragraph summary is going to be limited!  I've tried to follow the format of describing the limitations of conventional solutions, decrying from time to time idealistic approaches and then outlining some of the newer approaches arising from our work in naturalising sense-making.

Getting people to work across silos (formal)

One of the bigger issues, and one not best solved by committee.  We need to get real here and the reality of committees is that they are political.  Set up an cross silo committee and the people appointed will not be the natural networks, prepared to sacrifice sectional interest for the wider and common good; they will be people who are considered politically safe, able to defend departmental interests.  Now there is nothing necessarily wrong with this.  Having different perspectives and different interests is key to scanning range (its one reason I think it was a mistake for example to over integrate Homeland Defense in the US post 911).  Agent based models create competition between agents as a way of optimising the system overall (A good article on this in the current edition of New Scientist by the way).  It's a sort of Aristotelian Golden Mean where virtue is a half way house between the virtue not present and the virtue taken to excess.  Too much competition nothing happens, too little and you get bland conformity and group think.

One of the radical alternatives I and others are working on here is the concept of crews as a way of ritualizing, and formalizing cross silo activity.  A crew works because its members take up roles for which they are trained, and where their expectations of the other roles in the crew is also trained and to a large extent ritualised.  This means that people can assemble into a crew without the common forming, norming, storming & performing cycle.  A crew has cognitive capacity beyond the sum of its members, members occupy their roles for limited time periods, with people swapping between roles to allow for continuity.  In addition crews can delegate power in context outside of the normal hierarchies.  Now its early days here, but the way this should work is (i) identify the natural roles within any inter-silo function, (ii) train people from silos into those roles, and in expectation or role, (iii) in active use ensure that people from one silo are not replaced in rotation (which should never be more than eight hours, ideally four) by people from the same silo.  The whole point about rotation and time limitation here is to ensure the crew identity dominates over individual needs and interests (and yes that is a good thing in this context).

Getting people to work across silos (bottom up, top down stimulated)

Crews represent an up front investment in time, but will not always be appropriate.  In practice if people meet and work together they will build relationships that will persist beyond the activity in question.  It creates something I called trust tagging; you know the way you ask a friend a question and they get one of their friends to help out.  The common friend creates a trusted link.  The general evidence is that more than three degrees of separation means that trust breaks down.  It is also true that self-forming teams have higher degrees of self motivation than ones where people are allocated by management.  In the former case I would let down my mates if I don't pull my weight, in the latter case I can blame HR for the team creation.

One of the techniques we created in Cognitive Edge was social network stimulation, more fully described here.  The basic structure is simple, people are given the opportunity to collectively take on intractable but interesting projects for which success will reap rewards.  The catch is that they can only be won as a team, and the rules for team formation are determined to maximise new contacts and connections.  Are goal is not only to solve problems, but to reduce that critical degrees of separation number across the whole organisation to three or less.  Now this is very effective in cross silo working, not only in solving problems but in identifying those individuals who readily form relationships across traditional boundaries.  Running an SNS every six months for two years can result in a highly networked organisation, in which informal trusted networks can be used to link and connect people in the context of need.  Its about managing the ecology of knowledge flow rather than knowledge itself.

Sharing information

There are some basic truths here that we need to acknowledge.  More effort has been spent trying to get people to share knowledge and information (I will stay open on which is which here) across silos than anything else in government.  People have been complaining about the problem since the Greeks and its now going to change any time soon.  It's a basic truth that, asked for information in the context of a defined need few people will refuse it, but asked to share everything in anticipation of that need, they will refuse, or simply fail to co-operate.  Of course when we look at past failures, hindsight allows us to see ways in which fusing data or allowing multi-agency access to data, would have prevented a disaster , or allowed an opportunity to be seized.  In consequence it's a understandable that sharing and fusing are common knowledge management projects.

Now there is obviously some benefit in this, and technology capabilities make it easier every year, but its only ever a partial solution, and it can end up as a disaster.  Attempts to create common databases for the medical records of the British population are proving problematic  to say the least as the reality of linking and connecting confidential data emerge.  Now that is just within one department, namely Health.  Imagine the issues when you move across departments.  The other issue here is that of context, it's all very well having someone else's material but you have to understand what it means.  Semantic analysis while powerful, is limited compared with human sense-making capabilities (more on this later) and what really matters is to be able to connect very quickly with someone who can provide that context.  This means focusing on sharing meta-data rather than the data itself.  It's a lot cheaper and easier to generate metadata, and multiple types thereof, including query specific metadata sets (see later) than it is to fuse data, or deal with problematic issues of confidentiality and security.  If a pattern is found in the metadata, then specific requests can be made for access, and with that access comes human context.

Capturing lessons learnt

The mistake a lot of people make is to assume that people want a lot of messy experiential data synthesized into best practice documents and doctrine.  As previously mentioned the Iraq war showed the power of fragmented, experience based blogging without synthesis to communicate knowledge literally in the field under fire.  People spend hours on the internet, reading blogs and surfing in part because the material is fragmented, unstructured and encourages serendipitous discovery.  No one has to compel people into the world of social computing.  Now I have blogged in this subject many times and its also the theme of my column in KM World.  I am increasingly convinced that this is the key way forward for knowledge management activity, especially in government.

It makes the way forward on lessons learnt easier and while data intensive (the least of technology problems, storage is cheap, search not hard) it requires less management time and effort and critically is less demanding on employees.  Encourage blogging and the use of RSS feeds.  Start to capture peoples ideas, impressions and learning through audio and video recording as they do things; let them do it themselves don't send out the film crew.  SenseMaker™ with its lightly constrained approach to self-indexing was designed with this in mind, but you need a battery of social computing tools, messy in nature, following the patterns of human neurological evolution rather than trying to constrain humans to the autism of formal computing and over structured documents.

INTERVAL (or INTER-MISSION if you want some jargon)

Right, I am calling a halt there for the moment and posting.  Tomorrow (hopefully) I will pick this up again and deal with weak signal detection, cognitive bias and the deployment of expert knowledge.  Also distributed cognition and the see-attend-act model of sense-making.  I may also rant a bit about the challenge of anti-intellectualism and the fear of acting differently which plague this space.  For the moment I am going to have a drink and they go and pick up by award from the Academy of Management before going on to a much anticipated meal with old friends.