Grounding in computer-supported collaborative problem solving
|
|  Yüklə 1.41 Mb.
|
SubjectsTwenty pairs of subjects passed the experiments15. We recruited subjects opportunistically around us, mainly among our students. Some subjects knew each other, but in most pairs they had no experience of working together. We postulated that subjects which do not know each other very well will more actively build common grounds. The level of MOO experience was heterogeneous and had some impact on the variables we will study. Among the 36 subjects, we compare those with a medium or good experience of the MOO with the novices (respectively 16 and 20 subjects).
For other variables, we look at the global level of experience for the pair.
 Figure 2: Comparison of action commands used by novice versus more experienced MOO users
In summary, the difference of MOO expertise seems to have more affected the interaction among subjects than the problem solving actions. We will see that MOO dialogues include some events (parallel threads, mixed turns, ...) which may disorient novice users. The data presented above seem to indicate that the difference of time if more due to the difference is interaction frequency than to a difference with respect to problem solving behavior.
We present the observations in two stages. First, we draw the global picture of how grounding is achieved in this environment, progressively we focus on problem solving and on the whiteboard. The global picture is obtained by describing first the general rate of acknowledgment. We then review 3 specific dimensions of acknowdegment: delay, symmetry and co-presence. Then for relating grounding and problem solving, we caracterize grounding mechanisms according to the content (facts, inferences, management, ...) and to the mode (MOO dialogue / whiteboard / action). Different patterns of grounding (mode X content) will then be related to problem solving strategies. We end this section by addressing specifically the relationship between the type of representation on the whiteboard and the problem solving strategy.
We computed the rate of acknowledgment19, i.e. the ratio between the number of acknowledge interactions and the total number of interactions. Interactions includes MOO messages but also whiteboard actions and even actions in the MOO. We focus here on acknowledgment in MOO dialogues, i.e. a message say via 'say' or 'page' and acknowledged via 'say' or 'page'. We refer to these interactions in the remaining as 'talk/talk'. They represent 86% of all acknowledgment. We treat other modalities and multi-modal interactions in section 6.5. If talk/talk interactions, the pairs acknowledge in average 41% of the utterances. The distribution of acknowledgment rate (talk/talk) is bi-modal: we have five pairs in the range [28% - 35% ] and the remaining 13 pairs in the range [41% - 51%]. In the two experiments where subjects communicated by voice (pairs 1 & 2), the rate was 90% (respectively 88 and 92). This comparison is slightly awkward since the acknowledgment rate is dependent on the way speech is segmented into utterances: in MOO dialogues, the segmentation is performed by the users themselves who hit the 'return' key to send they message, while in voice interaction, we segmented ourselves the talk into utterances20. However, the difference of acknowledgment rate in these two conditions (MOO dialogue versus voice dialogue), 41% versus 90%, cannot be explained by the sole issue of coding. It does certainly reflect more a difference in the cost of grounding, as analyzed in section 3.1. Example 8 illustrates the low cost of voice interactions: while a simple acknowledgment, without any additional information, such as "yes", should not necessarily be acknowledged, it often is acknowledged in voice interactions.
Example 8: Acknowledging a simple acknowledgment (from Pair 1, translated) If a conversation includes only pure acknowledgment, i.e. non informative messages which do not bring any information more that the reception of the message, 50% would be a very high rate. At the opposite, if the acknowledgment is a dialogue move (Baker, 1996) such as a refinement, a counter-argument, etc..., it has to be itself acknowledged. We could hence return the point and infer that if a rate of acknowledgment is higher than 50%, it indicates that the acknowledgment was no a simple backchannel, but contained more information (refinement, refutation, ...). In other words, the rate of acknowledgment might indirectly a level of finesse in negotiation. The probability of acknowledgment relates to the form of the utterances being referred to. MOO messages tend to be sentences, while voice utterances are often subset of sentences, syntactically incorrect. More importantly, in voice conversations, a simple variation of intonation may be interpreted as a request for acknowledgment. In MOO dialogues, some sentences include an explicit request for acknowledgment, either because the whole sentence is a question or it is turned into a question by including an explicit hand-over messages as in the example 9.
Example 9: Request for acknowledgment (Pair 16) . There was not clear difference of MOO experience between pairs with a high acknowledgment rate versus those with a low acknowledgment rate (we do not compute means here since these are qualitative data). Moreover, the rate of acknowledgment is not directly related to frequency of talk: the 9 pairs who interact most frequently (number of 'say' and 'page' per minute) have almost the same average acknowledgment rate than the 9 other (respectively 0.41 and 0.42 messages per minute). This means that the differences in acknowledgment rate cannot be explain by some 'verbosity' variable such as typing speed or MOO expertise21. We compared the problem solving behavior of the 9 pairs with the highest acknowledgment rate versus the 9 pair with lowest rates. The groups do not differ with respect to global performance measures. In the group with low acknowledgment rate, 7 pairs found the right solution versus 6 in the group with a high rate of acknowledgment. The was not significant different in time, the average time being respectively 120 and 125 minutes for the low and high rates. We hence looked at finer measures of problem solving activity. We observed that pairs with a low acknowledgment rate perform significantly more actions in the MOO than pairs with a high acknowledgment rate. We count here the number of non-communicative MOO actions: ask, move, read, look, etc. The average number of actions is 237 for low rate pairs versus 178 for high rate pairs (F=5.13, df=1, p=.05). Moreover, pairs with a low acknowledgment rate show an higher redundancy rate than pairs with a high acknowledgment rate: 18 for the former, 6 for the latter (F=11; df=1; p= 0.01). This result is related to the former: when the number of actions is so high, subjects inevitably come to ask twice the same question. These results are rather surprising and interesting. Surprising because acknowledgment rate seems more related to problem solving variables that to dialogue variables. Interesting, because it confirms what we are looking for, a relationship between patterns of interaction and problem solving strategy. If pairs with a low rate of acknowledgment rate need more actions to reach the same solution (no difference in time or success), it may simply be that they are less efficient, less coordinated, in problem solving. We did also observe that pairs with a low acknowledgment rate perform fewer actions on the whiteboard than pairs with a high acknowledgment rate (average of respectively 92 and 125 actions). However, this difference is not significant (F=1.5; df=1; NS; F(.05)=4.5). If it was confirmed, this relationship would be difficult to interpret. We could set a an hypothesis that pairs 'low acknowledgment rate and few whiteboard interactions' are simply not very collaborative, subjects being not very attentive to each other. | ||||||||||||||||||||||||||||||||||