Course Information

• 472 611 Introduction to Assessment
• 472 612 Assessment Instrument Development
• 482 610 Introduction to Evaluation
  • On-Line:Introduction to Evaluation
• 482 611 Introduction to Research Methods
  • On-Line:Introduction to Research Methods
• 482 612 Evaluation Project

Assessment Strand

• 476 661 Assessment Design Analysis
• 476 662 Competency and Performance Assessment
• 476 664 Advanced Measurement and Psychometrics
• 476 694 Education Measurement, Assessment & Evaluation
• 476 695 Assessment and Reporting for Student Learning

Evaluation Strand

• 481-809 Program Evaluation: Forms and Approaches
  • On-Line:Program Evaluation: Forms and Approaches
• 481-810 Evaluating Large-scale Programs
  • On-Line:Evaluating Large-scale Programs
• 481-811 Evaluation for Management and Development
  • On-Line:Evaluation for Management and Development

Combined Assessment and Evaluation Strand

• 476-645 Multivariate Methods
• 481-812 Qualitative Methods
  • On-Line:Qualitative Methods In Evaluation
• 481-813 Quantitative Methods
  • On-Line:Quantitative Methods in Evaluation
• 481-814 Project in Assessment and Evaluation

472-611 Introduction to Assessment

Lecturer:
Points: 25
Semester:

This subject will examine the procedures associated with assessment and reporting in educational settings. Students will explore modes of data collection, recording mechanisms and interpretation frameworks. An in-depth study of the use of pupil assessment data, with specific emphasis on linking assessment results to the teaching and learning cycle and an examination of how this leads to tensions for accountability, will be examined. Practical examples of collecting and recording assessment data and its synthesis with external data sources will be presented. Collation of data to use for a range of purposes will be explored such that reports, their format and presentation is matched to the audience and information needs. A fieldwork exercise will be incorporated such that students examine both formal and informal data sets and devise methods of synthesis for a range of audiences and purposes.

This subject provides a conceptual overview of theories, issues and recent practice in evaluation. An inductive/case study approach is employed in which recent evaluation practice is analysed using frameworks developed by the lecturers. Topics covered include: conflicting views on the nature and purpose of evaluation; alternative purposes (summative, formative, illuminative); forms and approaches adopted by external and internal evaluators; evaluation and performance auditing; responsive evaluation; links between evaluation, decision-making and action; planning evaluations; an introduction to various qualitative and quantitative evaluation techniques in data management; reporting evaluation findings; evaluation ethics and standards; and a critical appraisal of selected Australian evaluation studies.

This subject will provide an introduction to the collection, processing, analysis and reporting of qualitative and quantitative data in evaluation studies. The subject begins with a consideration of the appropriateness of qualitative and quantitative methods in studies involving both small and large numbers of cases and general methodological issues of research design. For studies with small number of cases, topics will include participant observation, interviewing, focus groups and data analysis. For studies with medium or large number of units of analysis (eg surveys or enumerations) topics will include: sampling; design; development and testing of instruments for data collection, field processes; data processing; computer-based exploratory and descriptive analysis and general issues of validity and reliability. The quantitative component focuses on the development of skills based on intuitive rather than mathematical understandings.

472-611 Assessment Instrument Development

Lecturer: Nguyen Thi Kim Cuc
Points: 25
Semester: One

Students undertaking this subject will develop and trial two types of assessment instruments. In the first part of the subject an objectively scored instrument will be developed and trialled. In the second part a subjectively scored instrument will be developed. Students will explore the constraints that both forms of assessment impose and develop procedures for analyzing and interpreting data collected from both forms of assessment. Issues such as reliability and validity, and their impact on developmental scales and constructs underpinning the assessments will be explored. Implementation issues such as production and distribution, data collection and preparation will be undertaken as part of the fieldwork aspect of the subject.

482612 Evaluation Project

Co-ordinator: Rosalind Hurworth
Points: 25
Semester: One or Two

This subject is designed to enable students to apply knowledge acquired in previous subjects in the Postgraduate Certificate to a social or educational problem. Students are expected to apply this knowledge to a project of relevance to them in the workplace. The substantive content varies depending on the nature of the project.. It is intended that students should us process skills such as negotiating the project, managing informat-ion, using appropriate communication skills and being aware of political agendas amongst clients.

A good understanding of basic statistics will be assumed in this subject and the application of multivariate analysis methods will be a strong focus of the subject. Particular emphasis will be on such techniques as MANOVA, multi level modelling and covariance structure analyses (causal modelling) as well as on the interdependence of measurement, design, and analysis. Practical aspects of data management and analysis, literature searching, and report writing will be included in the laboratory component. The importance of theory underlying hypothesis testing will also be high-lighted. A "model comparison" approach will be used to enhance understanding of the links between theory, data and conclusions. Contrasts between traditional approaches and the model comparison approach will be illustrated through consideration of specific data sets.

Topics include: assessment strategies, assessment instrument design; domain specification and definition; test preparation, assessment task development; scoring approaches, strengths and weaknesses, item response theory (one, two and three parameter models) analysis, application, and interpretation of item response and classical analyses of data, using data to improve instruction, designing data bases, item banks, adaptive testing, assessment systems, competency based on assessment, test equating, and dimensions of learning.

This subject comprises a study of performance and competency-based assessment; assessment outside schools, industry, higher education, further education; recognition of prior learning (RPL) policies and practices; strategies for competency assessment; assessment for instrument and process design; assessment systems; technical issues of interpretation and application of performance assessment monitoring performance assessment systems; technical issues of interpretation and application or performance assessment; measurement model applied to performance assessment; application of item response theory to performance assessment and profiles and competency assessment of individuals and groups.

This unit is intended to provide an advanced understanding of item response modelling and psychometric theory. The subject examines item response theory from an advanced perspective, including the development of single and multiple parameter models, their specification, estimation and evaluation. It focuses on procedures for calibration and banking tasks based on rating and criterion referenced scales, constructed response and judgement-based assessments as well as multiple choice tasks. Computer applications linked to item response models and the other specific purposes in psychometric analysis include test equating and adaptive and on-line assessment.

This subject provides an overview of the nature of educational measurement, assessment and evaluation by contrasting various perspectives, purposes, models and procedures. It provides an introduction from a conceptual and practical viewpoint, covering such areas as: classical test theory; test bias, reliability and validity; item response theory; the design, implementation, analysis and evaluation of assessment procedures. Both qualitative and quantitative aspects of measurement are addressed and emphasis is placed on the appreciation of quality measurement through practical exercises involving the design and analysis of assessment tasks.

This subject focuses on the relationship between learning and assessment procedures and how, by building on this relationship, both learning and assessment may be improved. It will begin with an exploration of the varying purposes, principles and policies of assessment and reporting and consider trends and perspectives at local, systemic and national levels. Innovations in assessment and reporting, such as authentic, profiling and responsive and portfolio assessment will be investigated and evaluated in terms of their congruence with the promotion of student learning through the delivered curriculum. Shifting conceptions of validity and reliability that incorporate notions of equity, curriculum fidelity, manageability and public credibility will be examined. Ways that assessment practices can be utilised to strengthen the links between improved learning outcomes and quality and excellence will also be examined. Students will then apply the lessons of their critical evaluation of assessment and reporting to the design and evaluation of a particular instance of assessment and reporting of an educational program within their own context.

A conceptual overview of theories, issues and recent practice in evaluation. An inductive/case study approach will be used in which recent evaluation practice will be analysed using frameworks developed by the lecturers. Topics covered include: conflicting views on its nature and purposes; alternative purposes (summative, formative, illuminative); forms and approaches adopted by external and internal evaluators; evaluation and performance auditing; responsive evaluation; links between evaluation, decision-making and action; planning evaluations; an introduction to various qualitative and quantitative evaluation techniques in data management; reporting evaluation findings; evaluation ethics and standards; and a critical appraisal of selected Australian evaluation studies.

This subject encompasses particular generic skills. On completion of the subject students should be able to:

• demonstrate an understanding key evaluation concepts within the framework of delineating, obtaining and providing information;
• be able to plan a small scale evaluation;
• be aware of the recent developments in evaluation, particularly as they apply to the Australian context;
• identify the limits of evaluation in decision-making about social programs.
  and
  
• be able to undertake a critical review of an evaluation project on the basis of criteria provided

481-810 Evaluating Large-scale Programs
On-Line (Restricted Access)

Lecturer: Brad Astbury
Points: 25
Semester: One

This subject is based upon a critical analysis and application of the "theory-based" approach to evaluating large-scale education and social programs. Topics will include: the social and political context of large-scale interventions; explication of the "normative" theory of large-scale programs using concept mapping and related approaches; factors affecting program implementation in multi-level systems; research designs for evaluating large-scale programs; program monitoring and the appropriate uses of performance indicators; development and testing of "causative" program theories using quasi-experimental, causal modelling and case-study approaches.

The subject encompasses particular generic skills. On completion of this subject students should be able to:

• Possess highly developed insights into the nature of large-scale, multi-site, multi-intervention social and educational programs.
• Be adept at applying program theory and program logic approaches to the analysis and understanding of large scale programs, especially with respect to incorporating a range of diverse perspectives into evaluation activities.
• Be capable of choosing appropriate research designs for the evaluation of large scale programs, especially with respect to collection and analysis of evaluation data
• Have the methodological competence to conduct some elements in the evaluation of large-scale programs, particularly with respect to mixed method and theory driven approaches.
• Be sensitive to the social and political context of large scale programs and their evaluation.

This subject is based on an assumption that leaders and managers need timely systematic information for use in decision-making about programs [and policies] for which they are responsible. The focus is on internal evaluations within organisations [for example schools, government agencies and businesses] that have adopted a learning organisation focus.

Students will be introduced to program planning techniques such as program logic, and evaluability assessment. Other topics include evaluative techniques to develop and improve programs. These include: needs assessment; translation of policy into local action; development of policy from innovative practice; knowledge synthesis and various approaches to implementation including action research, benchmarking and empowerment evaluation. Students will also be exposed to a range of appropriate data management approaches and reporting drawing on case studies of current practice.

The subject encompasses particular generic skills. On completing the subject students should be able to:

• consider ways in which organisations operate when adopting a learning orientation
• appreciate the need for well articulated programs (and policies) and be able to develop them
• demonstrate an understanding of how evaluation can assist with the development and design of a new program, or monitoring and consolidation of an existing program
• review the impact of a given program in terms of its effects on the organisation as a whole.

481-812 Qualitative Methods
On-Line (Restricted Access)

Lecturer: Rosalind Hurworth
Points: 25
Semester: Two

Approaches to the collection, display and analysis of qualitative data will be examined, drawing on current evaluation and research in education and other social program areas. Topics to be covered include: issues associated with qualitative approaches (such as issues in fieldwork and how to maintain rigour); qualitative data collection options (such as interviews, focus groups, participant observation, the use of narrative; document analysis, use of visual media and case studies); methods of data reduction, display and content analysis, using both hand methods and appropriate computer packages; and ways in which findings may be reported to various audiences.

On completion of this subject, students should be able to:

• link research or evaluation questions to appropriate qualitative data techniques
• apply a range of core qualitative data collection techniques
• undertake simple analysis and reporting related to small-scale qualitative data sets.

This subject encompasses particular generic skills. On completion of the subject students should be able to:

• Understand the contexts in which quantitative methods are appropriate in research and evaluation, and recognize the suitability of particular quantitative methods in specific circumstances
• Be capable of developing simple research and evaluation designs for social and educational programs, including, where appropriate, activities such as sampling, data collection, data analysis and interpretation.
• Be adept at the use of the SPSS suite of computer programs in the analysis and display of data.

The subject is based upon an examination of various methods of collection, display and analysis of quantitative data. Topics will include: design of quantitative research strategies consistent with evaluation objectives; sampling; development and testing of data collection instruments; issues of reliability and validity; data processing; some descriptive and inferential methods of statistical analysis including multi-variate techniques; the use of standard micro computer packages for data management and analysis; the complementary use of qualitative and quantitative data in evaluation; and methods by which uni- and multi-variate quantitative data can be displayed and reported to various audiences.

The subject 'Project' in Assessment or Evaluation is designed to enable students to apply knowledge acquired in previous subjects in the Postgraduate Diploma in Assessment and Evaluation to a social or educational issue that is relevant to them in the workplace.

On completion of this subject students should be able to undertake, under supervision, a study from planning to completion. While the substantive content will vary from student to student, all will be expected to demonstrate certain generic skills such as to:

• negotiate
• be aware of the political agenda amongst stakeholders
• manage information (both at collection and analysis stages)
• use appropriate reporting skills.