Here’s a visualization of a conversation between five people trying to solve a murder mystery using a threaded-chat-like forum.
Each colored line represents a unique topic of conversation, and each position on the y-axis (there are 13) captures the evolution of each topic in time. As the conversation proceeds topics appear, disappear, and sometimes flow into other topics as people pull different pieces of information together. For example, the visualization shows that as people talk about topic 8, they draw upon information from topics 6, 7, 1 and 5 (in that order).
The links (thinner, translucent) make the connections between topics explicit. In the visualization, topic 1 flows into topics 8,9 and 10. Topic 8 flows into topics 10 and 13. Links carry a topic’s ancestry forward, so the link from 8 to 10 carries topic 1 with it.
The background along the lifespan of a topic displays the intensity of collaboration about that topic. Collaborative intensity combines a few things – the speed of posting, how balanced the conversation is, and how much attention we can assume team members are devoting to the topic (evidenced by lack of conversation elsewhere in the forum). The brighter blocks indicate periods of more intense collaboration.
The design is intended to draw your eye to those points in the conversation where topics converge and collaboration becomes more intense. There are four such points in the above image – (8,15), (10,26), (9,27), (13,28) – but it is not always the case that the two measurements correlate so well. Different conversations have vastly different characters.
At points where both topic convergence and collaborative intensity are high, conversation can look like this:
The “Aha!” moment is a moment of collaborative insight, where people put their different information together to come to a new understanding. It is a moment of collaborative knowledge creation. Groups don’t always have such insights, but in the data I’ve looked at, they always occur in passages of conversation like these.
The Topic Evolution Analysis (TEvA) Algorithm
Many techniques in information retrieval are based on the underlying notion that when people speak or write, unique ideas can be distinguished by analyzing the co-occurrence relationships of words. However, most of these techniques have been developed to support search, and have therefore focused on discriminating ideas, rather than understanding how they converge.
The TEvA algorithm is also based on the analysis of word co-occurence relationships, but I am primarily interested in convergence. The simple insight in my analysis is that we can view the evolution of knowledge much in the same way we can view the evolution of communities of people. In fact TEvA is for the most part a simple reapplication of Palla, et al.’s (2007) community evolution algorithm to networks of words instead of people. I’ll make a paper accessible with a more detailed description in the near future, and source code for others who want to try it out.
Why is this useful?
There are some really good techniques for analyzing collaborative knowledge construction / knowledge building that allow researchers to get a deep understanding of a group’s dynamics as it creates new knowledge. However, these techniques require hand-coded transcripts, and so cannot be readily applied to large volumes of loosely collaborative conversation. In this era of big data, we could benefit from a technique that could sift through data quickly to identify likely spots for creation of new knowledge.
Another limitation of existing approaches is that they focus almost exclusively on conversational dynamics and the discursive role of individual utterances rather than the knowledge being created. In taking a somewhat narrower view of knowledge creation, TEvA is able to generate a descriptor for newly created knowledge. This means that with some additional work, it might be possible to track this knowledge across intersecting social networks.
Why is it interesting?
I find the term “collective intelligence” inspiring, but also a little frustrating. It is such an evocative and imprecise term that it seems to actively seduce people into misusing it. Tom Malone provides a sound functional definition, but doesn’t in my opinion do much to clarify what precisely is meant by the word “intelligence” in this context.
I hope to do something with my work to make things a bit more precise, and TEvA is a stab at doing this. TEvA allows us to focus on the moments of interaction where collaborators are likely to create new knowledge, and it highlights the process leading to this kind of knowledge construction. Thus TEvA is a way of measuring the creativity of a collective. To my mind, this is a concrete step in the direction of making the term collective intelligence a bit more precise.