External Assessment in Austria

Präsentation zum Thema: "External Assessment in Austria"—  Präsentation transkript:

1 External Assessment in Austria
© Gabi FRIEDL- LUCYSHYN (2009)

2 Educational context: Very recent introduction of elements of external assessment: Monitoring of standards at levels 4 and 8 (June 2008) Baseline tests: May 2009 for lower secondary May 2010 for primary level Nationwide assessment: from 2012 on, one subject every year Centralised written school leaving exam (October 2009) Piloting since 2007 First nationwide assessment: 2014 So far internally assessed Matura Certificate (=leaving certificate at the end of upper secondary schooling

3 Standards on primary and lower secondary level
describe essential competences a student is supposed to sustainably have acquired in certain subjects at decisive stages in his/her school career formulate the desired outcome of the learning process aim at evidence-based quality assurance on classroom, school and system level competences that are essential for further development in the subject (montoring and dissemintation, of educational inicators paradigm shift

4 3 Functions of Standards
1. Accountability for outcome of teaching and learning 2. Individual coaching of students (diagnostic feedback) 3. System monitoring

5 Clear focus on teaching, not on testing
No influence on assessment of individual student Aims: gauging the degree to which expected goals are achieved  data for quality assurance teachers must assure the acquisition of defined competences and coach – as best they can - students individually by continually comparing defined goals with actual performance Support: diagnostic tools

6 The Assessments (regular monitoring)
Psychometrically validated tests Subjects, test cycles Structure of feedback Design of feedack: Context variables (questionnaires) allow for fair feedback (performance in relation to classes and schools in comparable contexts) Feedback given in as differentiated a way as possible Trained experts can be called upon to support schools in interpreting data and taking action data alone won‘t do the trick

7 Challenges Creating ownership
Provision of support measures and systems for teachers and heads Development and implementation of formal control mechanisms

8 School leaving exam

9 Context Basis: BIFIE has the mandate to design, develop, and implement scientifically validated, competence-oriented written exams for all types of upper secondary schools Subjects: Language of instruction, Modern Languages (CEF Levels B1 or B2), Latin/Greek, Maths Aim: Validity of exam and assessment of competences Key data: nationwide introduction 2013/14 (general secondary), 2014/15 (vocational colleges on secondary level) © FRIEDL-LUCYSHYN 2009

10 Architecture: Basic concept: 3 phases: Certificate:
Combination of individual student and school profile as well as standardisation in essential areas 3 phases: „pre-scientific“ paper (internal) 3 or 4 written examinations (external) 2 or 3 orals (internal) Certificate: Exam results in each part + study programme of previous years (subjects, number of hours, grades) © FRIEDL-LUCYSHYN 2009

11 Written exam (external)
No external correction, but keys and justifications, assessment criteria und cut scores are binding Various models of quality assurance under discussion Online help desk and hotlines

12 Piloting in all subjects until 2013/14
Voluntary participation Modern languages: since 2008 (reading, listening, language in use), 300 out of 350 schools on board; writing in 2011/12 Maths, language of instruction, Latin, Greek: from 2010/11

13 Development of tasks, items and test papers – general procedure
Test development by Austrian universities Cooperation with/review by (inter)national tertiary institutions Active and essential role of teachers as practitioners at all stages Integration of learners (as target group) in validation process Open-ended and close-ended item formats

14 How tests come into being (foreign languages)
Designed and developed by teachers and methodologists (university academics): Regional, professional and institutional diversity Training Validation:  Trialing of tasks and items (+ questionaires)  Processing, analysis and interpretation of stats (test-psychologists + psychometricians)  Item-moderation  Trialing ….  Standard setting (stakeholders)

15 How tests come into being (2)
Aim of validation process: Appropriateness of level of difficulty for student population Validity and quality of items Clarity of instructions Motivation University of Innsbruck Cooperation/Review: University of Lancaster (Prof. Charles Alderson, Dr. Rita Green): Training of item writers, item moderation, standard setting

16 The test paper in the foreign languages
Skills Listening Reading Language in Use Number of tasks 4 -5 tasks 4 - 5 tasks 4 – 5 tasks Test formate Multiple Choice Multiple Matching True/False Short answer questions True/False/Not Given Editing/error correction Multiple choice cloze Word formation © FRIEDL-LUCYSHYN 2009

17 The test paper in the foreign languages
Writing: Part 1: argumentative text (essay); compulsory Part 2: 2 tasks, student chooses one Text types: letter, report, article, composition Writings skills: argumentative, discursive, descriptive and narrative Assessment scales B1/B2 Recommendations for orals © FRIEDL-LUCYSHYN 2009

18 Thank you! © FRIEDL-LUCYSHYN 2009

19 Evaluation of exam results 1
Results compared to internal tests in final year 2009 Results compared to internal tests in final year 2008

20 Evaluation of exam results 2
Results compared to internal tests in final year 2009 Results compared to internal tests in final year 2008

21 Evaluation 3 (N-444)

22 Warum wird Lesen, Schreiben, Sprachbewusstsein auf 500 verankert
Warum wird Lesen, Schreiben, Sprachbewusstsein auf 500 verankert? Was kann man dann aus dem für die Schularten draus schließen? Sprachbew. ist Stärke der AHS?

23 CEF erklären

24 Ratingsheet erklären

25