Grounding in computer-supported collaborative problem solving

Research setting

1. Goals

2. The method

We selected the task and the computational support accordingly to this exploratory method. In laboratory experiment style, we would have select a toy problem, with a simple technological system. We chosen a (almost) realistic task which average duration is two hours. We chosen a standard technological system available for free on Internet. We do not pretend that the experiments are completely ecologically valid, since the subjects come just for an experiment and since the task was imposed. But the scale of the task and scale of the system are very close to the scales a similar applications in real life.

3. The task

   1. Criteria for task selection

• 
  Computer support. The task should be such that any task-specific actions or manipulations that must be performed to complete the task can be done in a pre-exisiting software package, or can be easily programmed.
  
• 
  Complexity. The task must be complex enough to generate interesting discussion and to require collaboration. It shouldn't be something so simple that one participant can easily just go off and solve it by himself without any thought.
  
• 
  Need for Planning. This is related to the previous criterion, but more specific. The task should be something where thinking about what to do (and therefore talking about it) can be useful, in addition to just doing it. The task should include significant aspects of problem solving, where the steps to be taken, or even the strategy for proceeding can be a fruitful object of discussion.
  
• 
  Potential for Misunderstanding. Since we are studying grounding, there should be some potential for grounding to fail and require repair. This is obviously related to the previous two criteria, but we also want the discussion to have some points of ambiguity where participants could have different ideas of what is being said/referred to/ etc.
  
• 
  Graphical Dimension. The participants should be able to manipulate either the solution itself or the process of arriving, in a graphical form. On the other hand, the solution should not require graphical presentation, since we are examining how grounding is affected by diagrams, not graphical reasoning by itself.
  
• 
  Joint goals. We want this to be a true collaboration rather than copperation or competition. There should be one joint goal for both participants.
  
• 
  Symmetry. The two collaborators should have equal abilities to act and equal knowledge about the task. While there will inevitably be some differences among individuals (namely with regard to MOO experience), symmetry in action is a condition for supporting collaboration (Dillenbourg & Baker, 1996)
  
• 
  Formalizability. The ultimate aim of the project is to actually design a computational system to be one of the collaborators, using information gained about human-human collaboration in the first phases. Thus, we need to pick a task that's not too "fuzzy", so that a computer collaborator would have a chance at somewhat normal interaction.
  
• 
  Feedback. The participants should have some way of determining when they have reached the solution sucessfully: either a logical criteria or some external mechanism should allow them to verify task completion.
  
• 
  Reality. The task should preferably be something fairly natural - that people might actually do and find useful, rather than a very artificial task that seems less like communicating over the computer and more like doing some "weird computer thing".
  
• 
  Fun. The task should be somewhat enjoyable if we want to find subjects on Internet.
  

Most of these tasks was lacking in one or more of the criteria listed above. Contrastingly, our selected task of murder myseries within the MOO was very good at most of these. Embedding the mystery in the MOO allowed easy computer support. The mystery itself, with 11 suspects, complex motives and alabis, and numerous rooms provided a complex task, with need for planning, and additionally potential for misunderstanding, both in terms of moo functions themselves, and ambiguity over some of the relationships (e.g., the husband) and names (Mr. Saleve). This task also does not require any graphical element for the solution, but has several dimensions of information which can be fruitfully represented graphically, including timetables, locations, relationships, and more conceptual information, such as arguments and current suspicion. It is fairly formalizable, since the information necessary to reason about the tasks could be written in a formal language to allow prolog-style theorem proving for performing the necessary inferences. While there is no direct feedback (and in fact this was sometimes a problem, that the partcipantsreached the correct answer, but were not absolutely sure they had reached it without considering more evidence), the logical considerations of only one suspect having the three criteria (see next section) allowed the two participants a criterion for deciding when they were finished. The task is also not strictly a realistic one, though the same kinds of inference and strategy considerations are common to a number of real-world diagnosis and problem solving techniques. The collaboration was fun for most participants, some even asking for more! The mystery was also designed to give a single joint goal, and the MOO also action symmetry.

2. Description of the task

The task is fairly complex since the Auberge include 11 people plus the victim and various objects which play a role in the inquiry: the murder weapon (the Colonel's gun), the ski instructor jacket in the victim's room, the painting located in the bar and its insurance contract in the private residence, an ’open hours’ note in the restaurant, the hotel registry and the phone log (list of phone calls from each room). The subjects can ask 3 kinds of question to any suspect: ask a suspect what he knows about the victim, what he did the night before and what he knows about the objects mentioned above. This makes a total of 66 questions to ask. All answers do not include information, sometimes the suspect say "I don’t know anything about this jacket".

• 
  Regarding the motive, 3 main tracks exist: (1) the husband (Giuzeppe Vesuvio) - wife (Mona-Lisa Vesuvio)- lover (Hans Wenger) - lover’s girlfriend (Heidi or Lucie?) square, with its different forms of jealousy, is revealed by the ski jacket and different suspect answers; (2) some answers reveal that professional jealousy is a motive for Rolf and Claire Loretan; (3) the insurance rip-off on a fake painting, the real motive, requires to find multiple information: the painting, the contract (often discovered very late because it is in a room with no suspect) and the fact that the victim recently learned that the painting was a fake (information given by the art student).
  
• 
  Regarding the opportunity to get the weapon, the detectives have to find (1) the gun and identify that it belongs to the Colonel - which is easy-, (2) when the Colonel was away from his room (between 8 and 9) and (3) who could steal the gun during that period. The last point implies checking the activities of the 11 suspects during the evening.
  
• 
  Regarding the opportunity to kill, the detective have to infer when Mona-Lisa was killed between 10 and 10.30, by comparing different answers with the information in the phone log. Then again, the detectives have to check the activities of the 11 suspects during the evening to find out who could be alone and kill.
  

• 
  He killed Mona-Lisa because she was his insurance agent and found out that the painting was a fake. Mona-Lisa learned that from the art student, Lisa Jones.
  
• 
  The gun was stolen between 8 and 9 p.m., when the Colonel Von Schneider was at the bar with the ski teacher. Oscar had the opportunity to steal it: he pretended that he stayed alone in the kitchen around 8.30, when his wife made a phone call, but actually Rolf Loretan went there to ask for a pill and there was nobody.
  
• 
  Mona-Lisa was killed between 10 and 10.30. 10 is the time of her last phone call, a call to the same number as the call she gave at 6pm. 10.30 is the time when she was found dead. All people left the restaurant at 10, but Oscar (the chef) only at 10.30, he had hence the time to kill Mona-Lisa.
  

The complexity of this task is more the information load⁸ (large number of facts to organize) than the intrinsic complexity of the relations to be inferred. This will impact on the way subject use the whiteboard: as a tool for storing and organizing information (group memory) more than as a tool for disambiguating information. There was no much ambiguity about the words used in the suspect answers. This was probably a design mistake, given our focus on grounding mechanisms, but ambiguous answers would have made the task intractable. Despite the spatial context, the solution of the enigma does not imply any spatial reasoning such as "Hans could not got from X to Y without crossing this room and meeting Rolf".

We tested the task (and the VCE) with two pairs of subjects. Some suspect answers have been changed because they lead to erroneous tracks

The correct solution was found by 14 out of the 20 pairs. The time for completing the task was in average two hours (123 minutes⁹). It varies between 82 and 182 minutes. The average time of failing pairs was almost the same (113 minutes).