Grounding in computer-supported collaborative problem solving

Co-presence and acknowledgment

It occur in the protocols that detectives meet in the same room when they have long discussions. In example 11, Sherlock accepts (78.7) such a proposition, and verifies that they actually are together (79.1) before to resume discussion.

Examples such as 11 are frequent at the middle of the protocols, when the subjects carry a first synthesis, and overall at the end, when they try to reach consensus. But, does it mean that in general, they tend to meet for acknowledgment? We therefore computed another rate, perpendicular to the former: among all acknowledged sentences, 56% are acknowledged when partners are in the same room. Interestingly there is a strong relationship between the rate of co-presence in acknowledgment and the rate of co-presence in general talk, as it appears clearly figure 3. In addition, we found no relationship between the rate of co-presence in acknowledgment and the general rate of acknowledgment (figure 4), nor the coefficient of spatial sensitivity in talk (see section 5.8.1).

5. Modality

3. 
   Table 3: Number of examples for different modes of acknowledgment (sum for all pairs). Action refers to Moo commands except 'say' and 'page', while talk refer to 'say' and 'page' commands.
   

1. Acknowledgment through action / of action

We consider first acknowledgment of action by action, for which we found only 2 examples. This type of acknowledgment requires a well-defined communication context and a spatial context within which action can be interpreted as any other speech act. In the example 12, the context is that the subjects agreed to go to visit Oscar in the kitchen. Hercule goes to the kitchen (117.2), but Sherlock does use the 'walk to' command. Instead, since he is informed at 117.2 that Hercule has left (Sherlock sees a message "Hercule has left for the kitchen"), he decides to follow him (117.7). Hercule is informed of Sherlock's action since he will see the message "Hercule has arrived".

The other examples of cross-modality involve talk and action. There are two conditions for this type of acknowledgment. The first condition is visibility: Hercule's utterance can be acknowledged by Sherlock action X if and only if Hercule sees that action X has been performed. In example 13, Sherlock can see that Hercule asks the question (9.6) as Sherlock previously invited him, since they are in the same room. In example 14, Hercule can see that Sherlock did the requested move because Sherlock arrives in Hercule's room. Visibility also implies that the action of one subject has an effect on the screen of his partner. This is the case for commands such as move, read, ask, etc.

In the MOO, like in reality, visibility is bound spatially: to see that one's partner asked a question, one has to be in the same room; to see his move, one has to be in the room being left (message "... has left for...") or in the arrival room (message "X has arrived"). Hence, visibility implies some co-presence, before, during or after action. Not-surprisingly, the rate of co-presence in acknowledgment through action is higher than in talk/talk interaction, 75% versus 55%. This point was actually explicit in our coding scheme (see section 5.8.2): If Hercule asks Sherlock to do something and that Sherlock does it, but that Hercule does not see it, we did not count Sherlock's action as an acknowledgment since it does not contribute to mutual knowledge about action. It is obedience, not acknowledgment. The same rules apply to action/talk acknowledgment as in example 15 (Sherlock say 'hi' to Hercule because he saw him arriving in the kitchen) or to more complex patterns as in example 16.

Because visibility is crucial in talk/action or action/talk acknowledgment, some subjects check whether this condition is fulfilled: in example 17, Sherlock checks that Hercule can see the same object; while in example 18, Sherlock checks that the MOO command 'ask' provides the same information to all characters in the same room. Reasoning on mutual visibility implies to know where the other agent is, since visibility is bound to space. Actually, the subject do not often use the 'who' command which informs about the position of agents in the MOO. This issue of mutual knowledge about position was addressed specifically in the thesis of L. Montandon (see section 8.1).

Reasoning on what one's partner can see is related to MOO expertise: it is because one agent is used to receive message X in condition Y, that he may infer that his partner will receive a similar message in a similar condition. This also imply that a computational agent able to interact in the MOO should have the ability to reason about which message was received by his partner. What is interesting in the MOO environment is that this ability relies on formal rules, easy to identify and to implement.

The second condition of acknowledgment by action is that the MOO commands enable the speaker to express the dialogue move that (s)he wants to make. Two dialogue moves that can only be expressed through MOO actions: in example 19, we see simple acknowledgment (84.8) and straight agreement -one agent suggests an action, the other does it (84.7). Therefore, the type of information being acknowledged through action is generally decision about actions, namely spatial moves, asking question and exchanging objects. The semantics of 'talking by action' are bound by the semantics of the MOO language. For instance, subjects cannot use MOO actions for negotiating who is suspect, because our experimental environment does not include commands conveying this type of information. We could include verbs (e.g. telling to a suspect that he can leave the auberge or putting a suspect in jail) or objects (e.g. putting and removing handcuffs or colored stickers) to indicate degree of suspicion. In other words, the design of the MOO commands (else than 'say' and 'page') defines a rather close semantic field, while the semantics of the talk and the whiteboard are widely open. Richer semantics can be expressed with the emote verbs ('grin', 'smile', 'frown'), but our subjects were not informed about these verbs (although a few advanced MOO users used them).

84.5			Priv	H	say Please type: give dn1 to Hercule
84.7	Priv	S		give dn1 to herc
84.8	Priv	H		say Thank you

Example 19: Talk/action/talk acknowledgment (from Pair 5)

Quite logically, the average delay in talk/action, action/talk and action/action acknowledgment is shorter than in talk/talk acknowledgment (34 seconds versus 48). On one hand, an action can generally be interpreted as acknowledging something, if it is produced very briefly after. On the other hand, since acknowledgment by action implies visibility and since often visibility implies co-presence, a delayed answer faces the risk that the partner has left the room before that the acknowledging action is performed.

In summary, the examples of acknowledgment through/of action are too rare to evaluate their cognitive impact. The conditions in which this form of acknowledgment may occur, both in terms of mutual visibility and in terms of semantics, depend on the design of the environement. An interesting direction of research is to design environments which integrate more action in dialogue and dialogue in action.

2. Acknowledgment around the whiteboard

We observed many examples of whiteboard /whiteboard acknowledgment. They can be classified in different categories:

• 
  Some subjects use the whiteboard for talking to each other as in the MOO. This is not frequent, except in pair 11, in which one subject has systematically problems with the 'page' command (more exactly with its abbreviation) and finally found easier to type interactions on the whiteboard (example 19)
  

• 
  A subject continues the action of his partner, for instance reuses a graphical code which has not been made explicit. In example 20, the subjects had drawn an map of the auberge on the whiteboard. Sherlock has been for a while pasting "done" labels on the map for indicating where he has already collected all information. Spontaneously, Hercule uses the same convention (31.8)
  

• 
  One subject disagrees with the content of a note put by the other: straight disagreement in example 20 (above) where Sherlock deletes Hercule note (31.9), refinement in example 21.
  

We consider now talk/whiteboard acknowledgment. Very often the information is presented in talk interactions and then put on the whiteboard by the same subject, and hence not counted here as a form of acknowledgment. We do not count either as talk/whiteboard acknowledgment the cases where Sherlock communicates some information to Hercule, who acknowledges it in the MOO and then puts it on the whiteboard. These interaction patterns (counted as talk/talk acknowledgment if there is acknowledgment) nevertheless indicate a concern for sharing information. Among the real cases of acknowledgment, we find several types of interactions:

• 
  The talk is sometimes simply an invitation to perform a particular action on the whiteboard, as in examples 22 and 23. These cases are isomorphic to the talk/action acknowledgment, some action being now performed on the whiteboard instead of through MOO commands, which implies in these experiments that the shared visibility condition is guaranteed.
  

• 
  The whiteboard is often used for archiving information which has been grounded through MOO conversations. In some cases, the agreement is explicit but the decision to write it on the whiteboard is implicit. In example 24, the archiving step is made explicit: after they have agreed that Heidi and Lucie were not suspicious anymore (73.7), they decide to archive that inference by drawing a red box around the respective notes concerning these two characters.
  

• 
  Deictic gestures would belong to the talk/whiteboard acknowledgment category. Our preliminary experiment on side-by-side collaboration (see section 8.2) revealed that 87% of gestures in front of a graphical display have a simple deictic function. Deictic gestures are absent from these protocols for two reasons: (1) the Sherlock's cursor was not visible on Hercule's whiteboard and vice-versa; (2) deictic gestures imply quasi-synchronicity between talk and gestures which is not possible in typed interaction, the average delay in talk/whiteboard and whiteboard/talk interaction being 70 seconds.
  

• 
  We observe again cases where the whiteboard is used for direct dialogue, as if in MOO talk.
  

• 
  We observe one example (26) of deictic: the words "this number" in Hercule's utterance (65.3) refer to the phone number that Sherlock just underlined twice on the whiteboard. Note that the delay here is 5 seconds, i.e. much less than the average delay in whiteboard-talk and talk-whiteboard interactions (70 seconds). Five seconds is a delay which could be accepted even with gestures associated to voice interactions. There are probably more examples of this kind, where the absence of cursor visibility is compensated by putting marks or moving slightly the object referred to. However, from a methodological point of view, these cases are very difficult to identify because they require a very fine grasp of the dynamics of interaction, a level of understanding which is not easy with transcripts.
  

• 
  We also observe cases where what is acknowledged is not the content of a note, but the action being performed (like in talk/whiteboard, talk/action, action/talk, etc.). In the example 27, Sherlock has quickly deleted a few name tags from the whiteboard, too quickly for Hercule, who wants to know whether Sherlock has deleted the name tag he previously put on the whiteboard.
  

• 
  The most important function of whiteboard/talk interaction regarding the elaboration of a shared solution is the negotiation of the information displayed on the whiteboard. A wide range of dialogue moves are possible since the acknowledgment is made by talk: simple acknowledgment (example 28), clarification of messages (example 29) or graphical objects (example 30), asking for justification (example 31).
  

In many cases, talk/whiteboard and whiteboard/talk acknowledgment form complex interaction patterns in which shared understanding is pursued in parallel in these two planes, as in example 32. This parallelism leads sometimes to synchronous communication, similar to those observed in other acknowledgment patterns (se section 6.6.2). In example 33, Hercule answers on the whiteboard (146.2), the question that Sherlock is simultaneously typing in the MOO (146.3)²⁵,.

When an inference is grounded before to be written down on the whiteboard, the whiteboard archives not only the inference itself, but also the fact that this inference has been agreed. Conversely, some times facts or inferences are first put on the whiteboard and then negotiated (or not). The balance of this two patterns depends on the global role of the whiteboard in problem solving (see section 6.8).

Like action/talk and talk/action patterns, talk/whiteboard and whiteboard/talk acknowledgment relies on the postulate that both subjects see the same area of whiteboard. This is not always the case when the whiteboard window is smaller than the whiteboard space. In these experiments, we fixed the space to the size of the windows in such a way that each partner could see what the other see. However, there remain two types of incertitude regarding the partner visibility: whether he is actually looking at the whiteboard or not (as in example 34 - requested acknowledgment) or whether, among all objects on the workbench, the partner sees a particular object (generally just created) (example 35 - non-requested acknowledgment).

24.3			Lobby	S	page Hercule do you look from time to time to the whiteboard?
25.2	r6	H		page s yes, I look at it!

Example 34: Grounding shared access (visibility) of whiteboard (from Pair 20, translated)

54.8			Lobby	S	draw red rectangle 23 398 387 457 [circling a suspect]
55.1	Priv	H		page s Yes I see....

Example 35: Grounding shared access (visibility) of an element on the whiteboard (from Pair 17)

The rate of co-presence in acknowledgment is lower (mean = 45%) for acknowledgment around the whiteboard (talk/whiteboard, whiteboard/talk and whiteboard/whiteboard) than in talk/talk acknowledgment. This may be due to the fact that the whiteboard is quite distinct from the MOO. Despite common technological roots, the whiteboard is displayed in a different window. These two planes of interaction do not seem to obey to the same spatial logic: when a map of the auberge is drawn on the whiteboard, it provides an external view of MOO space ("from the sky"), which contrasts with the immersive view ("from inside") provided in the MOO.

3. Grounding external references in different modes

The grounding of references vary according to the mode:

• 
  In MOO conversation, the reference is generally established by previous utterances (the context of conversation). The reference is not necessarily the last utterance, it is sometimes located several turns before.
  

• 
  In action/talk acknowledgment, the reference may be solved by co-presence. In example 37, Hercule uses "him" because he know that both detectives are in the same room, in which the only suspect to ask questions to is Oscar Salève. In example 38, the reference of "he" is also grounded by the fact that both detectives are in the same room (verified by a 'who') and have seen the same answers.
  

• 
  On the whiteboard, the external references are established by spatial criteria: row and columns in tables, simple proximity in less elaborated whiteboards, as in example 39. This example is interesting because the reference grounded in the whiteboard is reused in MOO dialogues, and because the abbreviation is progressive: first "the husband of ML", then "the husband" (twice) and then "he".