An Integrated Approach to Assessment of Higher Order Competencies

Staffing

P. Griffin, S. Gillis, J. Connally, (with K. Jorgensen, S. Oates, S. Kerr, L. Smith, P. Gillan, L. Reynolds, S. Wood, D. Cooper, D. McArdle, M. Tarrant, & R. Lee)

Summary

In collaboration with the Department of Defence, ACT and the Australian Emergency Management Institute (AEMI), the Assessment Research Centre explored an innovative approach to the assessment of higher order competencies in the public safety and public service industries by integrating developments in two fields of study: performance appraisals and psychometrics. The three-year study addressed an hitherto unresolved issue - how multiple sources of evidence about multiple components of competency judgments can be synthesised and used to inform holistic judgments of workplace competence. Two areas of investigation underpinned this study. The first was identifying a method of obtaining and synthesising data from multiple observers and the second was the method of separating the components of competency. Neither of these had been adequately addressed in the Australian Vocational Education and Training System, but each was pertinent to almost every industry sector and in many instances to other forms of distance education. This study explored how multiple sources of evidence could be synthesised and how components can be separated. It also examined the influence of the source of evidence (eg. peer, self and supervisor judgments) on overall decisions of competence.


Funding Source

Australian Research Council Strategic Partnership for Industry Research Training (SPIRT).

Australian Emergency Management Institute and Department of Defence - Joint Education and Training, Canberra.

Publications

Griffin, P., S. Gillis, Connally, J. & Jorgensen, K. (2005). An integrated multi-source measurement approach to the assessment of management competencies. Multi-Source Performance Assessments: Perspectives and Insights, ICFAI University Press.

Gillis, S., Griffin , P., Connally, J, Jorgensen, Oates, S, & Deftereos, C. (2004). Assessing Competencies at the Diploma and Advanced Diploma Levels: Evaluation Report, The University of Melbourne , Melbourne.

Assessment Research Centre and the Department of Defence (2003). Assessing Competencies at the Diploma and Advanced Diploma Levels. National Workshops, The University of Melbourne , Melbourne .

Bateman, A., (2003). The appropriateness of professional judgement to determine performance rubrics in competency based assessment. Master of Education thesis. The University of Melbourne , 2003.

Bateman, A., & Griffin, P. (2003). The appropriateness of professional judgement to determine performance rubrics in competency based assessments . Paper presented at the NZARE/AARE Conference, Auckland, December.

Connally, J. (2003) A multisource approach to the measurement of higher order competencies. Unpublished Doctorate of Philosophy thesis. Faculty of Education, university of Melbourne

Connally, J., Griffin , P., & Gillis, S. (2003). A multi source measurement approach for the assessment of higher order competencies. Paper presented at the NZARE/AARE Conference, Auckland , December.

Connally, J. Jorgensen, K., Gillis, S., & Griffin , P. (2003). A multi-source measurement approach to the assessment of higher order competencies. Paper presented at the Sixth Australian VET Research Association Conference “The changing face of VET”, Sydney , 9-11 th April.

Griffin , P., & Gillis, S. (2003). The assessment management information system. Computer software program, The University of Melbourne.

Connally, J., Jorgensen, K., Gillis, S., & Griffin , P. (2002). An integrated approach to the assessment of higher order competencies. Refereed paper presented to the Australian Association for Research in Education, Brisbane , December.

Wood, S., Gillis, S., & Cooper, D. (2002). Higher order assessment against emergency management competency standards. Paper presented at the Australian Association for Research in Education, Brisbane , December.

Gillis, S., (2002). Rural VET students. Directions in Education: Australian Council for Educational Administration, 11(7), 1-2.

Gillis, S., Griffin , P., & Pigdon, K. (2001). PARIS : Performance Assessment and Reporting Integrated System: Multi Media Project, The University of Melbourne .

Griffin , P. (2000). Competency based assessment of higher order competencies. NSW State Conference of the Australian Council for Educational Administration, Mudgee, NSW.

Griffin , P., & Gillis, S. (1999). Evaluation of vocational assessment processes and practices. Paper presented at the AARE NZARE Conference, Melbourne, November.

Griffin , P., & Gillis, S. (1999). Assessing higher order vocational competencies: A multi source approach. Paper presented at the AARE NZARE Conference, Melbourne, November.

Griffin , P., & Gillis, S. (1999). Discussion paper: The establishment of a competency based assessment system for emergency management. Assessment Research Centre, The University of Melbourne .

Griffin , P. & Gillis, S (1998). Competency based assessment of structured workplace learning. The Australian Journal of Vocational Education and Training in Schools, 1(1), 13-18.