External Assessment in Austria . © Gabi FRIEDL- LUCYSHYN (2009)

Educational context: Very recent introduction of elements of external assessment: Monitoring of standards at levels 4 and 8 (June 2008) Baseline tests: May 2009 for lower secondary May 2010 for primary level Nationwide assessment: from 2012 on, one subject every year Centralised written school leaving exam (October 2009) Piloting since 2007 First nationwide assessment: 2014 So far internally assessed Matura Certificate (=leaving certificate at the end of upper secondary schooling

Standards on primary and lower secondary level describe essential competences a student is supposed to sustainably have acquired in certain subjects at decisive stages in his/her school career formulate the desired outcome of the learning process aim at evidence-based quality assurance on classroom, school and system level competences that are essential for further development in the subject (montoring and dissemintation, of educational inicators paradigm shift

3 Functions of Standards 1. Accountability for outcome of teaching and learning 2. Individual coaching of students (diagnostic feedback) 3. System monitoring

Clear focus on teaching, not on testing No influence on assessment of individual student Aims: gauging the degree to which expected goals are achieved  data for quality assurance teachers must assure the acquisition of defined competences and coach – as best they can - students individually by continually comparing defined goals with actual performance Support: diagnostic tools

The Assessments (regular monitoring) Psychometrically validated tests Subjects, test cycles Structure of feedback Design of feedack: Context variables (questionnaires) allow for fair feedback (performance in relation to classes and schools in comparable contexts) Feedback given in as differentiated a way as possible Trained experts can be called upon to support schools in interpreting data and taking action data alone won‘t do the trick

Challenges Creating ownership Provision of support measures and systems for teachers and heads Development and implementation of formal control mechanisms

School leaving exam

Context Basis: BIFIE has the mandate to design, develop, and implement scientifically validated, competence-oriented written exams for all types of upper secondary schools Subjects: Language of instruction, Modern Languages (CEF Levels B1 or B2), Latin/Greek, Maths Aim: Validity of exam and assessment of competences Key data: nationwide introduction 2013/14 (general secondary), 2014/15 (vocational colleges on secondary level) © FRIEDL-LUCYSHYN 2009

Architecture: Basic concept: 3 phases: Certificate: Combination of individual student and school profile as well as standardisation in essential areas 3 phases: „pre-scientific“ paper (internal) 3 or 4 written examinations (external) 2 or 3 orals (internal) Certificate: Exam results in each part + study programme of previous 4 years (subjects, number of hours, grades) © FRIEDL-LUCYSHYN 2009

Written exam (external) No external correction, but keys and justifications, assessment criteria und cut scores are binding Various models of quality assurance under discussion Online help desk and hotlines

Piloting in all subjects until 2013/14 Voluntary participation Modern languages: since 2008 (reading, listening, language in use), 300 out of 350 schools on board; writing in 2011/12 Maths, language of instruction, Latin, Greek: from 2010/11

Development of tasks, items and test papers – general procedure Test development by Austrian universities Cooperation with/review by (inter)national tertiary institutions Active and essential role of teachers as practitioners at all stages Integration of learners (as target group) in validation process Open-ended and close-ended item formats

How tests come into being (foreign languages) Designed and developed by teachers and methodologists (university academics): Regional, professional and institutional diversity Training Validation:  Trialing of tasks and items (+ questionaires)  Processing, analysis and interpretation of stats (test-psychologists + psychometricians)  Item-moderation  Trialing ….  Standard setting (stakeholders)

How tests come into being (2) Aim of validation process: Appropriateness of level of difficulty for student population Validity and quality of items Clarity of instructions Motivation University of Innsbruck Cooperation/Review: University of Lancaster (Prof. Charles Alderson, Dr. Rita Green): Training of item writers, item moderation, standard setting

The test paper in the foreign languages Skills Listening Reading Language in Use Number of tasks 4 -5 tasks 4 - 5 tasks 4 – 5 tasks Test formate Multiple Choice Multiple Matching True/False Short answer questions True/False/Not Given Editing/error correction Multiple choice cloze Word formation © FRIEDL-LUCYSHYN 2009

The test paper in the foreign languages Writing: Part 1: argumentative text (essay); compulsory Part 2: 2 tasks, student chooses one Text types: letter, report, article, composition Writings skills: argumentative, discursive, descriptive and narrative Assessment scales B1/B2 Recommendations for orals © FRIEDL-LUCYSHYN 2009

Thank you! www.bifie.at © FRIEDL-LUCYSHYN 2009

Evaluation of exam results 1 Results compared to internal tests in final year 2009 Results compared to internal tests in final year 2008

Evaluation of exam results 2 Results compared to internal tests in final year 2009 Results compared to internal tests in final year 2008

Evaluation 3 (N-444)

Warum wird Lesen, Schreiben, Sprachbewusstsein auf 500 verankert Warum wird Lesen, Schreiben, Sprachbewusstsein auf 500 verankert? Was kann man dann aus dem für die Schularten draus schließen? Sprachbew. ist Stärke der AHS?

CEF erklären

Ratingsheet erklären