An Integrated Approach to Assessment of Higher Order Competencies
Staff
P. Griffin, S. Gillis, J. Connally, (with K. Jorgensen, S. Oates, S. Kerr, L. Smith, P. Gillan, L. Reynolds, S. Wood, D. Cooper, D. McArdle, M. Tarrant, & R. Lee)
Summary
In collaboration with the Department of Defence, ACT and the Australian Emergency Management Institute (AEMI), the Assessment Research Centre explored an innovative approach to the assessment of higher order competencies in the public safety and public service industries by integrating developments in two fields of study: performance appraisals and psychometrics. The three-year study addressed an hitherto unresolved issue - how multiple sources of evidence about multiple components of competency judgments can be synthesised and used to inform holistic judgments of workplace competence. Two areas of investigation underpinned this study. The first was identifying a method of obtaining and synthesising data from multiple observers and the second was the method of separating the components of competency. Neither of these had been adequately addressed in the Australian Vocational Education and Training System, but each was pertinent to almost every industry sector and in many instances to other forms of distance education. This study explored how multiple sources of evidence could be synthesised and how components can be separated. It also examined the influence of the source of evidence (e.g. peer, self and supervisor judgments) on overall decisions of competence.
Funding Source
Australian Research Council Strategic Partnership for Industry Research Training (SPIRT).
Australian Emergency Management Institute and Department of Defence - Joint Education and Training, Canberra.
Publications
Griffin, P., S. Gillis, Connally, J. & Jorgensen, K. (2005). An integrated multi-source measurement approach to the assessment of management competencies. Multi-Source Performance Assessments: Perspectives and Insights, ICFAI University Press.
Gillis, S., Griffin , P., Connally, J, Jorgensen, Oates, S, & Deftereos, C. (2004). Assessing Competencies at the Diploma and Advanced Diploma Levels: Evaluation Report, The University of Melbourne , Melbourne.
Assessment Research Centre and the Department of Defence (2003). Assessing Competencies at the Diploma and Advanced Diploma Levels. National Workshops, The University of Melbourne , Melbourne .
Bateman, A., (2003). The appropriateness of professional judgement to determine performance rubrics in competency based assessment. Master of Education thesis. The University of Melbourne , 2003.
Connally, J. (2003) A multisource approach to the measurement of higher order competencies. Unpublished Doctorate of Philosophy thesis. Faculty of Education, university of Melbourne
Griffin , P., & Gillis, S. (2003). The assessment management information system. Computer software program, The University of Melbourne.
Connally, J., Jorgensen, K., Gillis, S., & Griffin , P. (2002). An integrated approach to the assessment of higher order competencies. Refereed paper presented to the Australian Association for Research in Education, Brisbane , December.
Gillis, S., (2002). Rural VET students. Directions in Education: Australian Council for Educational Administration, 11(7), 1-2.
Gillis, S., Griffin , P., & Pigdon, K. (2001). PARIS : Performance Assessment and Reporting Integrated System: Multi Media Project, The University of Melbourne .
Griffin , P., & Gillis, S. (1999). Evaluation of vocational assessment processes and practices. Paper presented at the AARE NZARE Conference, Melbourne, November.
Griffin , P. & Gillis, S (1998). Competency based assessment of structured workplace learning. The Australian Journal of Vocational Education and Training in Schools, 1(1), 13-18.