Educational Data Mining Dialogue-Based Tutors (DBT) Referent: Paul Aurin Prof. Dr. Niels Pinkwart, Zhilin Zheng | SS 2014

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 2 Dialogue-Based Tutoring Systems AUTO TUTOR STUDIEN DISKUSSION IT SPOKE DEMO ANDES Kurzer Einblick (optional)

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 4 AUTOTUTOR

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 5 AUTOTUTOR Intelligentes Tutoring-System Einsatzgebiete: - Klassische Mechanik - informationstechnische Kompetenz - kritisches Denken Dialog mit Lernsoftware (Tutor) in Muttersprache des Lerners

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 6 AUTOTUTOR AUTOTUTOR stellt eine Reihe von Fragen oder Problemen Antworten erfordern eine verbale Erklärung. VISUALISIERUNGEN Abbildungen, Animationen, Videos werden eingesetzt ANIMATION DES TUTORS Sprechanimation, Gesten und Mimik werden unterstützend verwendet.

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 7 EMT Dialoge expectation and misconception tailored dialogue AUTOTUTORLEHRER Expectation and misconception tailored dialogue Erwartete gute Antworten: Liste von erwarteten Fehlkonzeptionen: ExpectationsMisconceptions Feedback

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 8 AUTOTUTOR gibt positives, neutrales, negatives Feedback schubst / aktiviert den Lerner („What else?“) gibt Hinweise verlangt Lücken zu füllen erstellt für fehlende Infor- mationen Be- hauptungen identifiziert und korrigiert falsche Antworten beantwortet Fragen gibt Zusammen- fassung der Antworten

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 9 Beispielfrage in der klassischen Mechanik When a car without head rests on the seats is struck from behind, the passengers often suffer neck injuries. Why do passengers get neck injuries in this situation?

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 10 3D

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 11 AUTO TUTOR

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 12 DEMO AUTOTUTOR

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 13 Demo AUTOTUTOR Could you at least try to give me an answer? That why, we’ll have something as a starting place to work from. Let’s try this together. When the neck provides a large force to accelerate the head, what could happen to the neck? STUDENT I don’t know. AUTOTUTOR When a car without head rests on the seats is struck from behind, the passengers often suffer neck injuries. Explain why!

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 14 Demo AUTOTUTOR Could you at least try to give me an answer? That why, we’ll have something as a starting place to work from. Let’s try this together. When the neck provides a large force to accelerate the head, what could happen to the neck? STUDENT What is a force? AUTOTUTOR I’ll take a crack at answering the question. Force is any influence that tends to accelerate an object; a push or pull; measured in newtons. Force is a vector quantity. Was that what you were looking for?

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 15 Demo … AUTOTUTOR See if you know the answer to this. When the head and body are moving at the same acceleration, they moving at the same ___________ ? STUDENT distance AUTOTUTOR Moving at the same rate.

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 16 Demo … AUTOTUTOR Okay, now, let’s recap. […] Now, let’s go onto another physics problem.

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 17 AUTO TUTOR

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 18 AUTOTUTOR | Studienergebnisse „AutoTutor achieves learning gains of approximately 0.8 sigma (nearly one letter grade) depending on the learning measure and comparison condition“ - Arthur C. Graesser, Fellow, IEEE, Patrick Chipman, Brian C. Haynes, and Andrew Olney

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 19 AUTOTUTOR | Studienergebnisse Sprachqualität (TURING-Test) STUDIE 1 N. Person and A. C. Graesser, “Human or computer? AutoTutor in a bystander Turing test,” in Proc. 6th Int. Conf. Intelligent Tutoring Systems, 2002, pp. 821–830.

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 20 AUTOTUTOR | Studienergebnisse Evaluation der pädagogischen Qualität STUDIE 2 A. C. Graesser et al., “Why/AutoTutor: A test of learning gains from a physics tutor with natural language dialogue,” in Proc. 25th Annu. Conf. Cognitive Science Soc., 2003, pp. 1–6. Über 1000 Studenten (in 10 Studien) absolvierten in der Physik einen Vor- Test zu einem Thema und arbeiteten im Anschluss mit AUTOTUTOR und wiederholten den Test im Anschluss. Je nach Studie haben sich die Leistungen um 0.20 – 1.50 Sigma verbessert (arithmetische Mittel 0.85 Sigma), α =.05 Im Vergleich dazu nutzten einige Studenten ein Lehrbuch (o.a. Quellen) und erarbeiteten sich das Thema selbstständig. Wiederholung des Tests: Verbesserung um 1.02 Sigma

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 21 AUTO TUTOR

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 22 ITSPOKE Intelligent Tutoring SPOKEn dialogue system

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 23 ITSPOKE

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 24 ITSPOKE - BEISPIEL ITSPOKE: I didn't understand. What force acts on all objects within close proximity to earth? STUDENT: uh the force of gravity ITSPOKE: Fine. Besides the Earth's gravitational force, are there any other forces acting on the person, keys, and elevator after the release? STUDENT: no ITSPOKE: Yep. Thus, keep in mind that after the release, the only force on the person, keys, and elevator is the force of gravity. If gravity is the only force acting on an object, will it be moving or staying still? STUDENT: it will be moving ITSPOKE: Fine. Not only are the person, keys, and elevator moving, they have only gravitational forces on them. When an object is falling and has only gravitational force on it, it is said to be in what? STUDENT: freefall ITSPOKE: Good. Thus, after the release the only force on the keys and man is the downward force of earth's gravity, so they are in freefall. Before the release, what is the relationship between the man's velocity and the keys' velocity? STUDENT: they're equal

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 25 ITSPOKE

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 26 ANDES - TUTORINGSYSTEM

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 27 ANDES - TUTORINGSYSTEM

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 28 ANDES - TUTORINGSYSTEM

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 29 ANDES

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 30 Diskussion In welchen Bereichen können Dialogue-Based Tutors Lehrer ersetzen? Welche Bereiche sind ungeeignet für Dialoged-Based Tutors? FRAGE 1 Welches Dialogue-Based Tutor-System würden Sie nutzen bzw. einsetzen wollen? FRAGE 2 Welche Nachteile haben Dialogue-Based Tutors-System in Bereichen, in welche diese unterstützend bzw. alleinig eingesetzt werden können? FRAGE 3

HU Berlin | Educational Data Mining: Dialogue-Based Tutors | Referenten: Paul Aurin, Maximilian Koch 31 Quellen A. Graesser, P. Chipman, B. Haynes, and A. Olney. Autotutor: an intelligent tutoring system with mixed-initiative dialogue. Education, IEEE Transactions on, 48(4): , Nov A. C. Graesser, K. Vanlehn, C. P. Rose, P. W. Jordan, and D. Harter. Intelligent tutoring systems with conversational dialogue. AI Magazine, 22:39-51, D. J. Litman and S. Silliman. Itspoke: An intelligent tutoring spoken dialogue system. In Demonstration Papers at HLT-NAACL 2004, HLT-NAACL-Demonstrations '04, pages 5-8, Stroudsburg, PA, USA, Association for Computational Linguistics. 3 K. E. Boyer, E. Y. Ha, M. D.Wallis, R. Phillips, M. A. Vouk, and J. C. Lester. Discovering tutorial dialogue strategies with hidden markov models. In Proceedings of the 2009 Con- ference on Artificial Intelligence in Education: Building Learning Systems That Care: From Knowledge Representation to Affective Modelling, pages , Amsterdam, The Netherlands, The Netherlands, IOS Press. 4

Jetzt wissen Sie mehr! Viel Erfolg damit und danke für die Aufmerksamkeit!