Internal and External Scripts: Studies on the Interplay of Discourse, Cognition and Instruction in Computer-Supported Collaborative Learning Frank Fischer Keynote at the International Conference of the Learning Sciences, Utrecht, NL, June 25, 2008

Major contributions of Armin Weinberger Ingo Kollar Karsten Stegmann Christof Wecker Jan Zottmann Kati Mäkitalo-Siegl COSSICLE European Research Team Heinz Mandl

Discourse activities are assumed to be related to cognitive processes of learning in “Spirals of Reciprocity” (Salomon & Perkins, 1998) Epistemic activities: How learners work on a knowledge construction task (Baker, 2002) Argumentation: How arguments are warranted and how argument sequences evolve in the discussion Social mode of co-construction/ transactivity or How learners built on the contributions of their learning partners (Teasley, 1997) (Weinberger & Fischer, 2006: Multidimensional coding scheme)

Construction of argumentation sequences (cf. Leitão, 2000) Argumentation and learning (e.g. Lund,Erkens, Baker,Andriessen,Schwarz) Construction of single arguments (cf. Voss, et al., 1993; Toulmin, 1958) Construction of argumentation sequences (cf. Leitão, 2000) claim data/evidence warrant argument counterargument integration

Does it work spontaneously?

A case-based online discussion environment (Weinberger, Stegmann & Fischer, 2003)

Obviously, there is a lack of knowledge on the side of the learners how to beneficially engage in peer discussions.

Collaboration Scripts Collaboration Scripts: activity programs that activate or assign roles and associated activities that help individuals to understand and to act in specific collaborative situations (in part: Schank & Abelson, 1975). Cultural and personal scripts on collaboration (“internal collaboration scripts”) Re-conceptualising “lack of knowledge”: Erroneous, lacking or heterogeneous application of (internal) scripts in computer-supported peer discussions.

External collaboration scripts Instruction: External collaboration scripts include scaffolds for activating appropriate “internal scripts” or for guiding, sequencing and coordinating different roles and activities in collaborative learning situations (see Kollar et al., 2006; e.g., O’Donnell, 1999; Dillenbourg 2002; Rummel & Spada, 2005; Schellens et al., 2007). External scripts supporting Zones of Proximal Development Students are supported by a set of procedural (software) scaffolds that aim at supporting active participation on a discourse level which is beyond what learners would accomplish spontaneously (i.e., with their internal scripts; King, 2007; Kollar et al., 2003)

Empirical research program on scripting Several experimental studies in three series on scripted discussions, embedded in the curriculum Studies with about 350 groups of 3 university students in the domain of educational psychology Studies with about 200 science education students learning in groups of two (high school level) Follow-up field studies in medicine, computer science, and educational science

Script for the construction of single arguments Claim Warrant + Data Michael suffers from an inefficient self-attribution in maths. He believes he is not talented due to failures. This means an internal stable attribution of failures. Qualifier Michael simply might be lazy or swamped with maths or suffer from bad instruction. Claim ... Warrant + Data Qualifier Stegmann, Weinberger & Fischer (2007). Scripting argumentative knowledge construction. ijCSCL.

Script for the construction of argumentation sequences

Example sequencing roles and activities: The „social script“ (peer critique script) Case Analyst: New Analysis Case Analyst: First Analysis Constructive Critic: Critique Constructive Critic: Critique Case Analyst: Reply Case Analyst: Reply Constructive Critic: Critique Constructive Critic: Critique

Online peer discussion (raw data)

OK! Lasst uns zum Fall Klassentreffen wechseln. Andrea: Ich hab irgendwie den Überblick verloren! Sind wir uns einig dass das ein internal stabiles Verhalten ist und das eine negative Attribution der Eltern vorliegt. Zusätzlich könnte man sagen, dass der Lehrer eine Reattribution fördern sollte (um das Verhalten zu ändern). Und natürlich auch bei den Eltern. Ben: >Ich hab irgendwie den Überblick verloren! Sind wir uns >einig dass das ein internal stabiles Verhalten ist und das eine >negative Attribution der Eltern vorliegt. Zusätzlich könnte >man sagen, dass der Lehrer eine Reattribution fördern sollte >(um das Verhalten zu ändern). Und natürlich auch bei den >Eltern. OK! Lasst uns zum Fall Klassentreffen wechseln. OT/FNN/BS R3/FBF/FRA R4/FBF/FRA R7/FBF/FRA R17/ERG/BS Therefore, our data consists of written discourses of these online discussions. On this foil you can see two messages within such a discourse. Andrea try to solve the problem and Ben agree with her solution and suggest to go the next problem without conflict-orientation. The first step in analysing is to segemnt the discourse. ---klick--- Some segments are easier to determine than other .... After segemnting, each segment will be coded on several dimensions QUOTED KOO/AKZ/PLA

Script component for the construction of argumentation sequences

Effects of specific script components on the processes of online discussions Parti- Epist. Argument Argument Transactivity. cipation Activity constr. sequence Participation Script component Epistemic Script component Argument construction Script component Argument sequencing Script component Social Script component

Effects of specific script components on the outcomes of online collaborative learning Individual level outcome domain-general domain-specific Participation Script component Epistemic Script component Argument Script component Argumentation Script component Social Script component Karsten, könntest du mir hier die Effektstärken angeben? Und ein Sternchen machen, wenn Replikation gelungen ist? Christof, könntest du die Effekte für Fading ebenfalls beziffern?

How do external collaboration scripts work (i. e How do external collaboration scripts work (i.e., change cognitive processing) ?

Some say, discourse and cognition are inseparable and basically the two sides of one medal Hidden below the surface: Cognitive activities

Discourse processes Cognitive processes Stegmann, Wecker, Weinberger & Fischer (2007). In: Proceedings of the CSCL conference 2007. Mahwah, NJ: Erlbaum.

A think-aloud study in asynchronous discussion (Stegmann et al., 2007)

Discourse and cognition in an online discussion Learner with low knowledge gains Weinberger, Stegmann & Fischer: Saturday 11.00 hrs, Room B

Learner with high knowledge acquisition

Example of specifying, assigning, and sequencing roles and activities: The „social script“ (peer critiquing script) Case Analyst: New Analysis Case Analyst: First Analysis Constructive Critic: Critique Constructive Critic: Critique Case Analyst: Reply Case Analyst: Reply Constructive Critic: Critique Constructive Critic: Critique

The orchestration of discourse and cognition in a scripted discussion Learner with social script: Analyst role

How do external and internal scripts interact? A study in science education (Ingo Kollar et al.: Friday, 15.15, “Blue”)

WISE Bell, Linn; Kollar, Fischer and Slotta (2005)

Computer-supported collaboration scripts for inquiry learning environments Kollar, Fischer and Slotta (2007)

Measuring the learners’ internal scripts on collaborative argumentation: Test (prior to collaborative learning phase): Protocol of a fictitious dialogue about a science topic Protocol included complete and incomplete arguments and argumentation sequences sensu Toulmin (1958) and Leitao (2000) Task: Identification of good and poor “argumentative moves” and giving reasons for that Median split procedure on the basis of individual point score

Interaction of internal and external scripts - Results External script supported the acquisition of knowledge on argumentation (i.e. fostering internal scripts; without hampering domain knowledge acquisition) Only the internal script was effective for the acquisition of domain knowledge Process analyses reveal: external scripts are only effective in writing phases, not during (quantitatively dominating) oral discussion phases. There, internal scripts are more effective.

Getting rid of (external) scripts: Fading How can external collaboration script components be faded out once appropriate internal scripts are accessible? (e.g., Pea, 2004; Renkl et al., 2005) Christof Wecker & Frank Fischer, Friday, 15.15 hrs, Room A

Condition of relevance Claim Argument Type of argument Type of claim Condition of relevance Application support Sequencing Counterargument

argument schemata: type of argument type of claim application support: after 2 counterarguments

Results on fading of scripts Simply hiding more and more of the script components does not help much Effect of fading can be increased by using collaboration: distributed monitoring supports learners to take over the regulation of their skill → Ongoing: longer-term study on fading of scripts in classroom setting

Interdisciplinary challenges: Specification and formalisation of (external) collaboration scripts Collaboration Script as a „boundary concept“ - addressed by the European Research Team „Cossicle“ The challenge: psychologically valid, educationally effective and technically re-usable external scripts

European Research Team „Cossicle“ (NoE Kaleidoscope) Stavros Demetriadis Pierre Dillenbourg Andreas Harrer Computer Science Education Päivi Häkkinen Pierre Tchounikine Psychology Team Leader: Armin Weinberger Frank Fischer

Components and Mechanisms Participants Groups Roles Activities Resources Mechanisms Components Fading Traversion Rotation Repetition Distribution Formation 1 2 3 Kobbe et al. (2007). Specifying collaboration scripts. International Journal of Computer-Supported Collaborative Learning.

An editor for external collaboration scripts Graphical modeling and editing of scripts (Harrer et al., 2006)

Conclusion Learning through online peer discussion typically needs support How to improve? Computer-supported collaboration scripts can indeed establish ZPDs; they have highly specific effects on processes and individual outcomes (but also side effects) External scripts can re-orchestrate the interplay of discourse, cognition, and instruction in peer discussions Complex interplay of internal and external collaboration scripts over time: External scripts promote knowledge on argumentation and the development of internal scripts; internal scripts are more effective for domain knowledge acquisition Effective fading (Davies, 2003; Pea, 2004) of external scripts is not a straight forward removal of script components - collaboration can be effectively used We suggest the scripting framework by Kobbe et al. (2007) as interdisciplinary joint reference to accumulate scientific knowledge as well as to represent effective script designs.

Research has partly been funded By Deutsche Forschungsgemeinschaft (DFG) By Noe Kaleidoscope (EU 6 FP) By the Federal State Baden-Württemberg (Strukturfonds) Most of the papers can be found in the TELEARN open archive

Fostering simulation-based learning in medical education with Collaboration Scripts Jan Zottmann, Peter Dieckmann, Marcus Rall, Frank Fischer & Tatjana Taraszow: Paper #, Session #, Date

Simulation-Based Courses with Video-Assisted Debriefing Hands-on simulation courses 2-3 active participants 8-10 live observing participants Ziel des Simulatorkurses: Erhöhung der Patientensicherheit Lernen mit Simulation: aktiv Beobachtungslernen: passiv - praktisch ohne Vorgaben und in nicht strukturierter Form Umgestaltung in eine aktivere Lernsituation & stärkerer Fokus auf CRM-Heuristiken möglich?

Kollar et al. (2004), Wecker et al. (2005) Findings of two classroom studies with CSCL: No difference between classroom setting and “lab” setting Explanations: CSCL environment are not flexible enough to be easily integrated into the overall classroom instruction Lack of appropriate classroom scripts Students hardly use the expertise of the teacher appropriately

Future Challenge: Orchestrating CSCL and classroom instruction Bringing the teacher back into the conductor’s role Towards an orchestration of scripts on small group level and classroom scripts (Wecker, Kollar & Fischer, in prep.) More adaptable/flexible scripts (e.g. Tchounikine & Dillenbourg, 2006) Automated analyses of peer discussion (Rosé et al., 2008)

An editor for collaboration scripts Harrer and Cossicle (2006): Based on joint framework for the specification of scripts Easy-to-use visual language for teachers and instructional designers Supporting different families of scripts (e.g., Jigsaw, Reciprocal...) Simulation of visual models  model-based predictions Compilable to standard e-learning description language, e.g. IMS/LD descriptions

Framework for the specification of collaboration scripts a common terminology as a basis for knowledge exchange and integration in research

Research question How can we get learners to take over the metacognitive control of their skill application as soon as the fading of the script starts? Use of collaboration! Hypothesis: only the combination of fading and distributed monitoring by a learning partner fosters the acquisition of the cognitive skill of argumentation (interaction effect)

The orchestration of discourse and cognition with collaboration scripts Collaboration scripts may lead to an alternative orchestration of argumentation in discourse, and cognitive processes Argumentation scripts in particular foster Argumentation quality in discussion individual knowledge on argumentation Learning in groups is only better than learning alone fifthe groups are appropriately (e.g., scripted)

Scripted Interaction current case: Asia current role: analyst current role: critic current role: critic

How should it work? Cognitive processing, e. g. self-explanations Individual acquisition of knowledge Collaborative discourse

Important reason: Ineffective patterns of interaction Learners do not discuss the content adequately Learners do not negotiate meaning but build consensus quickly

How to improve online peer discussions with collaboration scripts?

Script effects: Scaffolding interaction - changing rhythms of collaborative knowledge building

Fading: Getting rid of external scripts Research question: How can fading of script components be used to support learners to increasingly take over the metacognitive control of their argumentation skills?

External collaboration scripts in other domains

Main effect external script: F(1,87) = 10.20; p < .01; Eta² = .11 Effects of differently structured internal and external scripts on the acquisition of domain-general knowledge on argumentation Mean scores in the domain-gernaral knowledge on argumentation test So what were the results? Concerning the effects of internal and external scripts on the acquisition of domain-general knowledge on argumentation, we found a significant main effect for the external script. It helped both learners with low and with high structured internal scripts to acquire more knowledge about argumentation which you can see in that the second and the fourth column are higher than the first and the third. Experimental Conditions Main effect external script: F(1,87) = 10.20; p < .01; Eta² = .11

Effects of differently structured internal and external scripts on the acquisition of domain-specific content knowledge Main effect internal script: F(1,87) = 9.27; p < .05; Eta² = .10 Effects of differently structured internal and external scripts on the acquisition of domain-general knowledge (on argumentation) Main effect external script: F(1,87) = 10.20; p < .01; Eta² = .11 With respect to the acquisition of domain-specific content knowledge, we however get a completely different picture. Here, we did not find an effect of the external script but we found instead a significant main effect for the structuredness of the internal script saying that learners with high structured internal scripts acquired more domain-specific content knowledge than learners with low structured internal scripts. The high structured external script was not able to further support the acquisition of domain-specific content knowledge.

Quality of arguments in written discourse Medium quality High quality Low str. ext. script High str. ext. script Marg. significant main effect for external scripts (F(1,41) < 3.14; p = .08; Eta² = .07) Significant main effect for external scripts (F(1,41) < 23.86; p < .01; Eta² = .37)

Structural quality of arguments in overall discourse Medium quality High quality Number of arguments Low str. ext. script High str. ext. script Significant interaction effect (F(1,40) < 5.62; p < .05; Eta² = .12) Significant main effect for internal scripts (F(1,40) < 10.48; p < .01; Eta² = .21)

Scripting argumentative knowledge construction 1. The promises of argumentative knowledge construction 2. Does it work spontaneously? 3. Why should it work? 4. How to improve with collaboration scripts? 5. Educational Psychology meets Computer Science: Specification and formalisation of collaboration scripts 6. Challenges

Scripting argumentative knowledge construction 1. The promises of argumentative knowledge construction 2. Does it work spontaneously? 3. Why should it work? 4. How to improve with collaboration scripts? 5. Educational Psychology meets Computer Science: Specification and formalisation of collaboration scripts 6. Challenges

Automated discourse analyses Can computers help to analyse discourse? (a collaboration with Carolyn Rosé at CMU / Ph.D thesis Karsten Stegmann)

Script for the construction of single arguments Claim Warrant + Data Michael suffers from an inefficient self-attribution in maths. He thinks he is not talented due to failures. This equals an internal stable attribution of failures. Qualifier Michael simply might be lazy or swamped with maths or suffer from bad instruction. Claim ... Warrant + Data Qualifier Stegmann, Weinberger & Fischer (rin press). Scripting argumentative knowledge construction. ijCSCL

Learning through discussion Students are supposed to acquire knowledge and skills to participate in argumentative discourse in their field Regular lectures and seminars rarely provide opportunities for students to participate in high-level argumentative discourse Technology-enhanced learning environments might provide these opportunities with specific tools supporting students in learning from discussion Peers and technology can be part of a Zone of Proximal Development

Effects of scripts on the quality of arguments

Script for the construction of single arguments Claim Warrant + Data Michael suffers from an inefficient self-attribution in maths. He thinks he is not talented due to failures. This equals an internal stable attribution of failures. Qualifier Michael simply might be lazy or swamped with maths or suffer from bad instruction. Claim ... Warrant + Data Qualifier Stegmann, Weinberger & Fischer (in press). Scripting argumentative knowledge construction. ijCSCL

Formalization of the Arguegraph Script (see Dillenbourg & Jermann, 2007)

Mediation of learning outcomes: Learner or learning partners?

Mediation of learning outcomes: Learner (r = .41) learning partners (r =.17)

Mediator Analyses Own quality of argumentation in discourse X knowledge acquisition on content (attribution theory): r = 0.30* Quality of argumentation of learning partner X knowledge acquisition on argumentation r = 0.11, n.s.

Results Scripts can facilitate collaborative learning beyond individual learning  it may be sometimes better to let students learn alone then in unstructured collaboration 82

Interaction of internal and external scripts Processes: Evidence of negative interaction Outcomes: internal scripts are much more effective than external: skills of argumentation help you learn content knowledge; maybe wm capacity problem with external scripts and complex phenomenon (Ingo fragen)

Hypotheses for Question 1 Additivity hypothesis: Interaction hypothesis: High strctd. High strctd. Low strctd. Low strctd. Internal Script Internal Script Low strctd. High strctd. Low strctd. High strctd. External Script External Script

Specifying Activities Critiquing the analysis

Summary of results so far Computer-supported collaboration scripts can change interaction dramatically and highly specifically - with side- effects Computer-supported scripts can indeed establish Zones of Proximal Development: Students can discuss quite beyond their current competence levels. Through scripted discussions, students improve their knowledge on argumentation. Only some of the scripts facilitate domain knowledge

The orchestration of discourse and cognition in a scripted discussion Learner with social script: Critic role

Script for the construction of single arguments Claim Warrant + Data Michael suffers from an inefficient self-attribution in maths. He believes he is not talented due to failures. This means an internal stable attribution of failures. Qualifier Michael simply might be lazy or swamped with maths or suffer from bad instruction. Claim ... Warrant + Data Qualifier Stegmann, Weinberger & Fischer (2007). Scripting argumentative knowledge construction. ijCSCL.

Script for the construction of arguments R1 R2 R3 R4 R5 R6 R7 R8 R9 Application of new knowledge in discourse Open discourse t R1 R2 R3 R4 R5 R6 R7 Application of new knowledge in discourse Script for the construction of arguments Depth of Cognitive Processing (Stegmann et al., 2007)

Mediator Analyses Argumentation in discourse is a mediator between argumentation scripts and knowledge acquisition on argumentation as well as knowledge on content.

Method Structuredness of the external collaboration script Low High Participants: 90 students (= 45 dyads) from five classes of two German secondary schools (grades 8 to 10) Learning environment: Module of the inquiry learning environment WISE about “Deformed Frogs” Learning partners collaborated in front of one computer screen 2x2-factorial design: Structuredness of the external collaboration script Low High Structuredness of the internal argumentation-specific scripts N = 20 (10 dyads) N = 22 (11 dyads) N = 26 (13 dyads)

Results - Individual Elaboration Kontrolle der Lernvoraussetzungen: Bedingungen unterschieden sich nicht im Vorfragebogen zur Kompetenzeinschätzung. Bedingung „mit Skript“ elaborierte sign. mehr zu Heuristiken als Bedingung „ohne Skript“; Bedingungen unterschieden sich nicht bzgl. Elaborationen zu Medizin. U-Test Elaborationen zu Medizin: U(-0,80) = 145,00, n.s. Elaborationen zu Heuristiken: U(-5,30) = 10,50, p < 0,001

Results - CRM skills Bedingung „ohne Skript“ schätzte ihre Kompetenz im Vorwissen bereinigten Nachfragebogen besser ein als Bedingung „mit Skript“!

Results: Knowledge about argumentation fading x distributed monitoring: F(3; 56) = 6,80; p = 0,001; h2 = 0,267