Skip past navigation to main part of page
 
MGSE Home : Search : Sitemap
Skip past navigation to main part of page
---

1998 Annual Report

On this page:
Charter of the Centre for Program Evaluation
Recent Key Developments
Highlights of 1998
Commissioned Program Evaluations
Teaching and Coordination of Graduate Courses in Evaluation
Developmental/Community Activities
Details of Staff Activities

Charter of the Centre for Program Evaluation

The Centre for Program Evaluation (CPE) is an evaluation and research centre in the Faculty of Education at The University of Melbourne. The objectives of the CPE are to:
  • act as a focus for the practice and theory of evaluation.
  • undertake commissioned program evaluations on a consultancy basis across a wide range of policy areas
  • teach and coordinate graduate courses in evaluation and contribute to research methods in the Faculty
  • provide developmental activities such as specialised training courses, for clients outside the University.
The core staff of the Centre includes an Associate Professor and Director (Dr John Owen), three senior lecturers (Dr Gerald Elsworth, Mr N Day and Dr Rosalind Hurworth), and a Centre Manager, Ms Marion Brown. Core staff are supported on by adjunct staff, graduate students and, from time to time, through secondments from other organisations.

Recent Key Developments

To meet these objectives the CPE has
  • made contributions to evaluation theory and practice via made major contributions in key evaluation journals and via evaluation conferences in Australasia and overseas.
  • adopted approaches to evaluation practice designed to serve the needs of its clients in the public and private sectors. These clients come from a range of areas including welfare, crime prevention and health, in addition to the educator sector, broadly defined to include both school education and training in a range of settings.
  • developed an articulated post graduate program in evaluation designed for evaluation practitioners and those who commission evaluation in their work.
A major initiative has been the use of the internet to offer evaluation subjects to students in distance mode. As of April 1999, five of the eight subjects offered in the Post Graduate Diploma in Evaluation/Master of Education are available to students via the internet. Each subject site provides a set of integrated learning sequences backed up by essential readings. Students contact lecturers via email and use of an interactive forum. This is the only known integrated post graduate program in evaluation world wide which is offered by distance mode online. While the offering of these subjects is still in the infancy stage, students from Canada, New Zealand, Hong Kong, Indonesia have enrolled, as well as others from remote locations in Australia such as Darwin and Townsville. Experiences to date have emphasised the need to manage ongoing contacts with students in addition to expertise in translating learning sequences into appropriate web formats.

The CPE has been a unit within the Department of Education Policy and Management [EPM] since the beginning of 1995. The CPE continues to enjoy excellent relations with EPM. In addition to offering a specialist teaching and supervision program in evaluation the CPE makes a major contribution to the teaching of research methodology offered by EPM on behalf of the Faculty.

At a broader level, there are ongoing contacts with groups outside the Faculty with similar interests. Some examples include:
  • research studies with outside agencies, for example with staff from the Australian Council for Educational Research
  • cooperative projects with other evaluation centres in Australia, for example the Institute for Social Programme Evaluation, Murdoch University
  • cooperative arrangements such as the secondment of a CPE staff member, to the Centre for Health Program Evaluation, in the Department of Public Health and Community Medicine during 1998 (Mr Neil Day)
  • workshops for professional bodies such as the Australasian and American Evaluation Associations
  • full time secondment of staff from other organisations. During the past three years, a vice principal from the Department of Education, and an auditor from the Victorian Auditor General's Office have been seconded to work on major evaluation projects
  • the use of the World Wide Web to link with evaluators world wide
  • student involvement in CPE based post graduate supervision and subject enrolment from across the University.

Highlights of 1998

An important strategy has been to consolidate the scholarly reputation of the Centre and to increase its profile among the international academic community.

There has been an increase in the number of reports, monographs and refereed papers produced by CPE staff, often based on the evaluation projects such as those listed below. A feature has been the increase in submissions by staff and students, to journals devoted to evaluation theory and practice, such as
  • the Evaluation Journal of Australasia and Evaluation News and Comment (in Australasia)
  • American Journal of Evaluation and Evaluation and Program Planning (in the United States), and
  • Evaluation (in the United Kingdom).
CPE staff also contributed notably to forums devoted to new directions in applied research and evaluation. Papers were given at international conferences held in Belfast, Warwick (UK), Rome, Chicago, and Melbourne, and invited presentations were given by staff in Goteborg (Sweden), Paris and Melbourne. Invited evaluation workshops were given at the Australasian Evaluation Society conference in Melbourne and at the American Evaluation Society conference in Chicago. A significant indicator of the influence of the CPE in the Melbourne conference was the level of involvement of present and past students of the Centre.

During 1998
  • Gerald Elsworth visited Sweden to give lectures and invited presentations, as part of a staff exchange arrangement involving the Universities of Melbourne and Goteborg. As part of this exchange we will welcome Dr Lena Lindgren from the University of Gotegorg during second semester, as a visiting fellow to the CPE.
  • John Owen, worked with Dr Patricia Rogers from RMIT University, to complete a major book on evaluation titled Program Evaluation, Forms and Approaches. The book has been published (1999) in Australasia by Allen and Unwin, and in the United Kingdom and North America by Sage Publications.
  • Rosalind Hurworth successfully submitted a Doctor of Philosophy thesis to LaTrobe University. In addition Rosalind visited the Centre for Educational Development, Appraisal and Research (CEDAR) in Warwick as part of a long term link between CEDAR and CPE.
  • Neil Day was seconded to the Centre for Health Program Evaluation to provide leadership and advice on evaluation theory and practice. He has maintained some supervision of graduate students in CPE and finalised a major evaluation for the non-government disability sector. This was a report of a study with the assistance of John Owen and Nick Myrianthis, from the Victorian Auditor General's Office.
  • A major trend is an increasing load of individual supervision by CPE staff at masters and doctoral levels. Staff are sought as supervisors both within the Faculty and University wide, for their evaluation expertise. In 1998, Lena Sanci, supervised by Neil Day and Professor Glen Bowes, (Centre for Adolescent Health) won an award from the US Society for Adolescent Medicine for the best paper presented at the annual conference by a new researcher. Lena is the only non American to win this award. The paper reported the design and findings of an evaluation of a training program in the practice of adolescent medicine by general practitioners.

Commissioned Program Evaluations

Staff of the Centre were involved in a range of program evaluations for clients outside the University during 1998.

Projects ranged in size and complexity, and spanned the fields; education and training, health, medicine, and criminology. Most of these evaluations were won by competitive tender application. It should be noted that, with the increase in the number of consulting organisations in the community, competition to win contracts is now intense.

Successful projects included:
  • an analysis of Year 12 curriculum provision in Victorian Government schools, for the Office of School Review, Department of Education in Victoria
  • an evaluation of the National Early Psychosis Project, as part of the National Mental Health Strategy
  • contribution to a national study of numeracy and literacy in primary schools funded by DEETYA
  • the review and development of Evaluation Models for Community Crime Prevention, for the National Campaign Against Violence and Crime (with the Institute of Social Programme Evaluation, Murdoch University)
  • a planning evaluation study, for Collingwood College
  • an evaluation of the use of information technology at Melbourne Girls Grammar
  • the development and review of a tailored professional development program in evaluation for the New Zealand branch of the Australasian Evaluation Society, in Wellington, New Zealand
  • a review of the Web Skills program for the Office of Further Education and Training.

Teaching and Coordination of Graduate Courses in Evaluation

The CPE is one of a small number of academic units world wide offering an articulated graduate teaching program in evaluation. The program represents a major contribution to the evaluation expertise of the professional community and attracts students from government agencies, the helping professions and from independent consultancy groups. Almost all students are involved in important program development and/or hold policy review positions.

The CPE presently staffs a Post Graduate Diploma in Evaluation, and a Master of Education with a specialisation in evaluation. In addition, staff of the Centre offer supervision on evaluation topics for PhD candidates, an evaluation seminar in the Doctor of Education program, and supervision within that program.

As indicated earlier the CPE has developed distance mode online offerings which enable students to study without attending the University. Experience with using this mode with a limited number of students has shown that staff need to adapt more conventional teaching approaches to accommodate the needs of students offsite. There are also resource implications associated with the development and maintenance of student learning. The CPE will monitor the implications of this mode of teaching during 1999.

Staff of the CPE also teach evaluation subjects within a range of other graduate programs, and are in demand to teach specialised evaluation subjects and short courses on methodology at other tertiary institutions.

In 1998, the total teaching load of the CPE was the equivalent of over 3.0 staff. This includes contributions to the research methods components of higher degrees where the CPE makes an important contribution. The increase in formal teaching loads has led to a shift in the role of core staff towards evaluation field work, that is, from an active hands on mode to one of providing leadership and strategic advice to project officers.

Developmental/Community Activities

The CPE was involved in a range of developmental activities designed to serve the needs of the evaluator and allied professions during 1998.

This included a major professional development training program for evaluators in New Zealand during second semester. The program was delivered in Wellington by John Owen and Ros Hurworth with some developmental assistance from Ms Julie McQueen. The program was organised around a core evaluation subject, Program Evaluation; Forms and Approaches. The 50 students enrolled in this subject through community access (CAPS) and had access to the website for the subject. An evaluation of the training which was built into the design showed that the program had high impact on most participants.

The Centre continued to be represented on professional bodies. Ros Hurworth was a member of the organising committee for the international conference of the Australasian Evaluation Society, held in Melbourne in October. Ros is also on the organising committee for the Australian Qualitative Research Association International Conference, Melbourne, in July 1999.

Internally, staff of the CPE worked on a wide range of Departmental and Faculty committees. Ros Hurworth was a Departmental representative on the Faculty Library Committee and Ros and Gerry Elsworth had major inputs to the Faculty wide research methodology offerings.

In addition John Owen chaired an evaluation working group which commissioned an evaluation of the Commonwealth Supported Accommodation Assistance Scheme, for the Department of Health and Family Services. This involved overseeing all phases of the evaluation process, including the selection of the evaluators, their methodological processes, and reporting of findings to stakeholders.

Staff of the CPE

Academic Staff

John M Owen, B Sc (Hons), MEd [Melb] Ph D [Monash]

John Owen's interests include the evaluation design, innovative approaches to data management, the use of evaluation in decision-making and change within organisations, and the teaching of evaluation.

Neil A Day, BA (Hons), Dip Ed, MA [Melb], Dip Data Anal [Essex]

Neil Day's interests include survey research, quantitative analytical techniques, and the use of computers in evaluation. Neil is currently on secondment to the Centre for Health Program Evaluation, a joint centre of Monash and Melbourne Universities.

Gerald R Elsworth, BSc PhD [James Cook] MAPsS

Gerald Elsworth interests include research design applied to program evaluation, survey research, quantitative data analysis, the management and reporting of multi-site and multi-method evaluation studies and program evaluation in higher education

Rosalind E Hurworth, BA (Hons) [Swansea] Post Grad Cert Ed [Nottingham] MEd Post Grad Dip Soc [La Trobe]

Rosalind Hurworth's interests include the use of evaluation in management and development, and in a range of qualitative techniques such as interviews, participant observation and focus groups, as well as various approaches to analysis.

Adjunct Staff (1998)

As indicated earlier, the CPE has worked cooperatively on projects with adjunct staff, defined as those who work on CPE funded projects.

In 1998, Julie McQueen assisted Ros Hurworth and John Owen in the development of a case study for the New Zealand evaluation training project.

John McLeod, who has previously worked on several CPE projects, worked with John Owen on an evaluation of the Web Skills project, undertaken for the Office of Training and Further Education.

Susan Day worked extensively with Neil Day on a major health review project during 1998. Susan has worked on a range of projects since completing the Post Graduate Diploma in Evaluation at CPE.

Jenni Livingston and Pamela Andrew were members of a team which undertook an evaluation of the use of information technology at Melbourne Girls Grammar.

Sergio Fabris, as part of his Teaching and Research Award, assisted with the development of new teaching programs taught by the CPE.

Management of the CPE

Marion Brown BBus (RMIT), is responsible for a wide range of management activities associated with the work of the Centre. In addition to what might be regarded as conventional management support, Marion has also taken a lead role in the trialling and implementation of a range of innovative technologies associated with research and teaching. These include the use of End-Note in report writing, the use of Mac Spirs for the development of annotated bibliographies, and the use of HyperResearch for the analysis of qualitative data.

During 1998, Marion consolidated CPE material on the internet. This included the CPE website and further development of evaluation subjects available on the web. Marion has also acted as first point of contact for students enquiring about enrolment and for distance students who have technical difficulties in accessing CPE subjects. This 'first point of call' role is significant in terms of effective public relations between the CPE and its student clients.

Facilities

Staff and students in the Centre have access to excellent computing facilities in the Faculty of Education. The CPE provides computers and a range of statistical and report generating software including the Statistical Package for the Social Sciences, SAS, Data Desk, EQS, JMP, Nudist and HyperResearch; specialist programs for qualitative and advanced quantitative analysis, and a variety of data base and word processing programs.

In addition, the Centre has the support of the University Library with access to social science bibliography data bases on interactive compact disk and a full set of ERIC material in microfiche. The CPE also has its own specialised library of program evaluation texts and journals.

Details of Staff Activities

Publications and Productions

Journal Articles

Elsworth, G. R. (1998). School size and diversity in the senior secondary curriculum: A generalisable relationship? "Australian Journal of Education", 42, pp. 183-203.

Elsworth, G. R., Harvey-Beavis, A., Ainley, J, & Fabris, S. Generic interests and school subject choice. "Educational Research and Evaluation", (in press).

Gijsbers, A., Dunt, D., Hurworth, R. et al. (1998) An Evaluation of the Cervical Cancer Screening Program. "Health Promotion Journal of Australia", 8, 2, 92-99.

Huang, N., Hurworth, R. & Dunt, D. (1998). Evaluation of the Victorian Cervical Cancer Screening Program 1995: A balancing act between pragmatism and rigour. "Evaluation Journal of Australasia", 10, 1-2, 69-81.

Owen, J. M and Lambert, F. C. (1998). Evaluation and the Information Needs of Organisational Leaders. "American Journal of Evaluation", 19, 3, 355-365.

Owen, J. M (1998). Towards a Theory of Negotiation in Evaluation. "Evaluation News and Comment", 7, 2, 32-35.

Owen, J. M (1998). Towards an Outcomes Hierarchy for Professional University Programs. "Evaluation and Program Planning", 21, 315-321.

Smallwood, H. & Hurworth, R. (1998). Literature Syntheses for Program Planning and Policy Development: A Guide. "Evaluation News and Comment", 7, 1, 37-44.

Invitational Lectures Seminars etc.

Elsworth, G. R. (1998) Australian higher education: Lurching towards the market? Paper presented to a staff and graduate student seminar, School of Public Administration, University of Gs¹teborg, October.

Owen, J. M. (1998). Training Evaluators to Support School Based Management. Paper given in the seminar series of the International Institute for Educational Planning, Paris, October.

Owen, J. M. (1998). Is the Logic of Evaluation Still Relevant? Invited Major Address at the International Conference of the Australasian Evaluation Society. Melbourne, October.

Conference Presentations

Day, N. A (1998). Ensuring Accountability by Safeguarding against Loss of Control. Paper presented at the Fourth Annual Contracting Out Conference, Millenium Hotel, Sydney, May.

Day, N A. (1998). Seminar: Piloting of a diary instrument for the Coordinated Care Trial. Paper read at CHPE Seminar, April, 1998, at CHPE, Heidelberg Hospital

Elsworth, G. R., Toohey, D., & Harvey-Beavis, A. (1998). Demand for Tertiary Courses in Australia and Ireland. Paper presented to the Annual Meeting of the British Educational Research Association, Belfast, Northern Ireland, September.

Hurworth, R. (1998). An Evaluation of a School Nurse Program in Victoria, Australia: Making Policy from a Single Case. Paper presented at the CEDAR Conference, Case Study in Education, University of Warwick, UK, March.

Owen, J. M. (1998). Interactive Evaluation: Using Professional Evaluators to Encourage Effective Participatory Evaluation. Paper given at the third annual conference of the European Evaluation Society, Rome, October.

Owen, J. M. (1998). Using Evaluation Forms to Structure a Post Graduate Program in Evaluation. Paper given at the third annual conference of the European Evaluation Society, Rome, October.

Owen, J. M. (1998). Towards a Theory of Negotiation. Paper given at the annual meeting of the American Evaluation Association, Chicago, November.

Owen, J. M. (1998). Linking Benchmarking Techniques to Principles of Evaluation Practice. Paper given at the annual meeting of the American Evaluation Association, Chicago, November.

Owen, J. M (1998). Fundamentals of Program Evaluation. Workshop given at the annual meeting of the American Evaluation Association, Chicago, November.

Evaluation Reports

Batterham, R., Fabris, S., Elsworth, G., Southern, D., Appleby, N., Dunt, D., & Liaw, T. (1998). Integrating GPs with the wider health care system. Development of a GP integration index: Methods and results after the concept mapping phase. Melbourne: The University of Melbourne, Department of Public Health and Community Medicine, General Practice Evaluation Program.

Collins, J. & Hurworth, R. (1998). Continuity of Care for Hepatitis B Carriers. Centre for Program Evaluation, University of Melbourne, December.

Day, N.A., Myrianthis, N., and Owen, J. M. (1998). Evaluation of the Brokerage Model of Coordinating Training Services to the Non-Government Disability Sector. Melbourne. Non-Government Disability Unit, Government of Victoria, ISBN 0 646 32720 8.

Elsworth, G. R. (1998). Data for Year 12 curriculum provision benchmarks. Consultant's Report for the Office of Review, Directorate of Education, Victoria.

Harvey-Beavis, A. and Elsworth, G. R. (1998). Individual Demand for Higher Education: Interests and Fields of Study. Canberra, ACT: Evaluations and Investigations Program, Higher Education Division, Department of Employment, Education and Youth Affairs.

Hill, P.W., Hurworth, R. & Rowe, K.J. (1998). The Place of Literacy and Numeracy in the Primary School Curriculum. Report for DEETYA. Australian Primary Principals' Association and the Centre for Applied Educational Research, The University of Melbourne, November.

Hurworth, R. (1998). Future Directions for Collingwood College. Centre for Program Evaluation, The University of Melbourne, January.

Hurworth R. (1998).Report on Focus Groups Concerning a New Pork Product. Report for Food Science Australia. Centre for Program Evaluation, The University of Melbourne. September.

Hurworth, R. (1998 and 1999). An Evaluation of the Dunlop Asia Leadership Program. Report for AsiaLink. Centre for Program Evaluation, The University of Melbourne. Interim Report July 1998. Final Report January, 1999.

Livingston, J., Owen, J. M. and Andrew, P. F. (1998). The Computer Enhanced Curriculum. Use of Notebooks in the Middle School Curriculum. Report to Melbourne Girls Grammar School, September.

Joyce, C. & Hurworth, R. (1998). Evaluation of the National Early Psychosis Project. Report for the EPPIC and the National Mental Health Strategy. Centre for Program Evaluation, The University of Melbourne, May.

McLeod, J. Owen, J. M. and Grogan, P. (1998). An Evaluation of the Pilot Phase of the Webskills Program. Report to the Office of Further Education and Training. [http://www.otfe.vic.gov.au/qcmd/webskills/eval/index.htm]

Owen, J. M. (1998). Evaluation Models for Community Crime Prevention. Stage III Report to a Steering Committee of the National Campaign against Violence and Crime. December.

In addition, Neil Day was involved in the preparation of the following reports at the Centre for Health Program Evaluation

Richardson, J., Hawthorne, G., Day, N. A. and McNeil, H. (1998). Interim Report to Victorian Health Promotion Foundation: Research Grant: Adapting the Assessment of Quality of Life Instrument for Health Promotion, Ref 96-0914. Melbourne: Centre for Health Program Evaluation.

Segal, L., Day, N. A., Day, S., Dunt, D Hawthorne, G., Piterman, H. and Robertson, I. (1999). Report to the National Evaluator, Coordinated Care Trials: Local Evaluation of the Southern Health Care Network Coordinated Care Trial: Mid Term Report. Melbourne: Centre for Health Program Evaluation.

Research Grants

Funded Projects

Elsworth, G. R.
Analysis of Year 12 Curriculum Provision in Victorian Government Schools, 1997. Consultancy to the Office of Review, Directorate of Education, Victoria.

Owen, J. M
Evaluation of Models for Community Crime Prevention.
Evaluation of Information Technology Curriculum at Melbourne Girls Grammar.
Development and evaluation of a tailored professional development program in New Zealand. With R. Hurworth.

Hurworth, R. E.
Evaluation of the National Early Psychosis Project.
Planning for the future at Collingwood College.
Consultancy for Food Science Australia.
DEETYA national study of Numeracy and Literacy in Primary Schools.
Consultancy for Hepatitis B study.
Various workshops on qualitative methods in evaluation.
Development and evaluation of a tailored professional development program in New Zealand. With J. Owen.

Student Supervision

Day, N.A.

Theses in progress:

PhD full time
Lena Sanci (joint supervision with Professor Glen Bowes). Ph D part time

Claire Lincoln (joint supervision with Dr Fiona McCulloch).

Master of Education, part time
Katherine Robertson

Elsworth, G.R.

Theses completed:

Sundram Sivamalai, "Needs analysis of a basic nurse education course in a regional community.", PhD.

Josephina Ho, "Effect of congruence in measured interests on self perceived academic competence in Australian Year 12 students.", MEd Minor Thesis.

Theses in progress:

PhD - Full-time

Sergio Fabris
Christine Matthews (joint supervision with Prof. B. Start)
Sue Wilkes (joint supervision with Dr Lee Emery)

PhD - Part-time

Roy Batterham (joint supervision with Assoc. Prof. David Dunt, Centre for Health Program Evaluation, application pending)
Delwyn Goodrick (associate supervisor, Dr R Hurworth)
Adrian Harvey-Beavis (associate supervisors Dr J Ainley, Dr F Naylor)
Angela Herd
Beverley Jackling (associate supervisor Assoc. Prof. G Burrows, Accounting)
Hitomi Mizuno (joint supervision with Dr Erica Frydenberg)
Maureen O'Sullivan
John Roodenburg
Ken Smith

MEd Minor Thesis - Part-time

Denise Findlay

Hurworth, R.E.

Theses completed:

Susan Turale. "Living Below the Poverty Line: A Phenomenological Study of the Experiences of Students of Education at the University of Ballarat". DEd (associate supervisor John Owen)

Graeme Harvey "Participatory Evaluation: Recording One School's Experience in Using Evaluation as a Process to Support Development." MEd minor thesis. (associate supervisor John Owen)

Theses in progress:

PhD-Part Time

Delwyn Goodrick (associate supervision with Gerry Elsworth)

MEd Part Time

Jilane Matinga
Laurie Ramsom (associate supervision with Irene Donohoue Clyne)
Rhonda Gordon-Brown (major thesis)

Owen, J.M.

Theses completed:

Susan Turale. "Living Beyond the Poverty Line: a Phenomenological Study of the Experiences of Students of Education at Ballarat University." Doctor of Education. With R. Hurworth.

Graeme Harvey. "Participatory Evaluation; Recording One School's Experience in Using Evaluation as a Process to Support Development." Master of Education Minor Thesis (with R Hurworth).

Judy Parker. "Evaluation of the Seminar Training Program at the National Disaster Training School. Master of Education minor thesis. Submitted.

Lyn Griffin. "Doctor and Patient Communication: Communication in Health Service Complaints and Complaint Resolution. Master of Education". Master of Education minor thesis.

Theses in progress:

PhD (part-time)

Kevin Fell. Reconciling Developmentally Appropriate Practice and Literacy.
Andrea Argirides. Evaluation of Mother and Baby Units as Organisations. (application phase).

Doctor of Education (part-time)

Carl Stevens. Quality Assurance in Catholic Schools.
Jenny Teng. Use of Program Logic to Implement an Innovation in Home Nursing.
Pam St Leger. Evaluability Assessment of a Vocational Education and Training Program
Elizabeth Pittman. Relative Effectiveness of Onsite and Distance Modes of Teaching in Nurse Education.
Faye Kohn. Influence of Evaluation on Decision Making by General Practitioners.

Master of Education (part-time)

Jenni Livingston. Program Logic of a Nurse Education Program.
Frank Quinlan. Evaluating the Impact of New Technology in Distant Locations.
Judy Forster. Evaluation of Innovations in Literacy in Queensland Schools.
Chris Jenkins. Evaluation of the Impact of Educational Technology in a Private Girls School.
Anne Markiewicz. Development of a Theory of Negotiation in Evaluation.
Tess Tuohey. Development of International Benchmarks in Ambulance Service Provision.
Ping Sun. Strategies for Encouraging Use of New Knowledge in Bureaucracies.
Elizabeth Symonds. Evaluation of Advisory Services in Health.
top of pagetop of page

Contact Us

Contact the University : Disclaimer & Copyright : Privacy : Accessibility