The survey was piloted in child and family agencies in the northern sector of the LSMR. It was modified as a result and then distributed to child and family agencies across the region. A total of 97 agencies and individuals working in child and family services received the survey. Of these, 76 surveys were returned. The surveys were anonymous and confidential; participation was voluntary.

Along with the training needs analysis, respondents were sent promotional material, which aimed to

The circulation of this material acted as reassurance to the participants that the time and energy they were putting into responding to the survey was likely to have a positive outcome (see Appendix 4, ‘Promotional Material’).

Additionally, the rural project coordinator promoted the Children’s Mental Health Tele-education Program project at health and welfare network meetings. The project was directly promoted to regional Child and Adolescent Mental Health Service (CAMHS) staff who were asked to pass on information about the Children’s Mental Health Tele-education Program to health and welfare workers they had contact with.

Training Needs Survey results

The most significant findings of the needs analysis were related to

Figure 2 (see Appendix 1), ‘Difficulties in attending training in child mental health’, demonstrates that time was the most significant barrier for these workers. Cost, travel and knowledge of training opportunities were also barriers. The lower rating given to arranging backfill may reflect the difficulties in rural areas of attracting locums.

Figure 3, ‘Access to training and information in child mental health’, shows that over 60 per cent of respondents rated their access to training and information in child mental health as ‘poor’. Approximately 5 per cent rated their access as ‘very good’; no respondent rated their access as ‘excellent’ (see Appendix 1).

The majority of respondents rated their current knowledge of children’s mental health as ‘poor’ or ‘satisfactory’; no respondent ticked excellent. As can be seen in Figure 4, ‘Current knowledge of child mental health disorders’ (see Appendix 1), over 80 per cent of respondents rated their current knowledge of child mental health disorders as ‘poor’ or ‘satisfactory’; no respondent rated their current knowledge as excellent.

The findings in regard to barriers to attending professional training were consistent with those outlined in the literature review. The results of the training needs analysis confirmed unmet need in the LSMR for professional development for allied health workers in child mental health.

Results from the training needs analysis regarding program design and content influenced the design of Children’s Mental Health Tele-education Program.

Interest in the program

The Children’s Mental Health Tele-education Program was a new and original program in a highly specialised area. It involved the innovative use of video technology and demanded a large commitment of time from agencies and staff. For these reasons, the program brochure included clear information on the use of the technology, the program content, time commitment, and the required discipline background for participants (see Appendix 4, ‘Promotional Material’).

In total, fifty applications were received from across the region and they were representative of a range of children’s service providers.

The program was funded to offer twenty places. Requests for a place in the program were received from the following agencies.

Table 1 Requests for program places by agency and location

A decision was made to offer thirty places in the program. In making this decision a balance had to be struck between the needs of workers for this type of professional development and maintaining the educational effectiveness of the program. A group of thirty was not considered ideal for a tele-education program yet at the same time it was difficult to turn workers away in the knowledge that they would be unlikely to access this training elsewhere in the region.

In allocating places, every effort was made to ensure equitable representation of professionals from a range of children’s services across the region. Those who did not gain a place were put on a waiting list.

The level of response to the Training Needs Survey and the Children’s Mental Health Tele-education Program validated the assumptions underpinning the project, that is,

A collaborative project

A Memorandum of Understanding outlining the responsibilities of each service in the implementation of the project was exchanged between the two services.

The following table indicates the distribution of responsibilities across the two services.

Table 2 Distribution of responsibilities

MHSKY provided program coordination and a consultant was engaged to provide independent advice in the development of an evaluation model.

A key feature in the development of this project has been the strong collaborative relationship between Royal Children’s Hospital Mental Health Service (MHSKY) and Bendigo Health Care Group Psychiatric Services. The relationship grew out of pilot projects that both services were involved in and which have been previously discussed. Throughout the period of development of innovative work using videoconferencing technology, a strong sense of trust developed between project managers in both organisations.

Adam and Walker (2000) have studied the role of trust as a key issue in collaborative relationships between individuals and organisations. Trust has been identified as a key element in successful interorganisational relationships and in facilitating joint action between organisations. Trust, it has been argued, is indispensable for any collaborative alliance to be formed and to function.

An important assumption underpinning the project was that this relationship was developed enough to successfully facilitate a sophisticated project such as this, involving as it did, the diverse elements of technology, distance, specialist clinical input and education.

Project aims and objectives

The Children’s Mental Health Tele-education Program aimed to deliver a high-quality, needs-based group professional development program in order to

Target audience

This project was targeted at allied health workers employed in child and family agencies in the LSMR. Participants were from a range of disciplines, including occupational therapy, psychology, nursing, social work and welfare work. The majority of participants were direct service providers.

Program development

The Children’s Mental Health Tele-education Program was developed with input from a number of levels, including allied health workers in rural agencies (through the training needs survey) and consultation with clinical staff at Bendigo Health Care Group CAMHS and Royal Children’s Hospital Mental Health Service.

The training needs analysis identified that the following topics were of most interest to allied health workers.

Topics were presented in a developmental framework on the advice of child mental health clinicians. This approach ensured that participants were sensitised to early warning signs of these emotional disorders in children and to the importance of using developmentally appropriate interventions.

Developmental framework topics included

As well, a session on Attention Deficit and Hyperactivity Disorder was included to cover problems that appear to have a more organic base. The resulting program covered the following topics.

The Children’s Mental Health Tele-education Program comprised eight sessions, consisting of four half-day sessions and four full-day sessions. All sessions were presented using interactive videoconferencing.

Presenters

In planning the program, it was recognised that this was a large commitment for agencies to make. As well, because the technology was a relatively new teaching medium, it was potentially a barrier to participation.

To attract participants to the program it was important to offer a level of expertise that rural health professionals could not easily access. If the program was to retain participants, teaching quality was critical. Presenters were therefore selected not only on the basis of their clinical expertise but also on their ability to teach using the medium.

Clinical staff from Royal Children’s Hospital Mental Health Service (MHSKY)made up the four presenters.

Program format

Presenters were based in Melbourne and the sessions were transmitted from there to the audience in Bendigo. The transmission was split by linking a data projector to the video equipment. The audience watched a presenter on the monitor and the PowerPoint presentation on a separate screen. This replicated the usual experience of a presenter, physically located in the room, speaking to a PowerPoint presentation.

A number of challenges presented themselves. The large size of the group imposed limitations on the amount of interaction that occurred between the audience and presenters. There were some practical limitations, too, such as a particularly sensitive microphone having to be passed to participants who wanted to have input into the presentation. Also, many people find it difficult to be vocal in a large group and probably even more challenging to interact with a presenter via the medium of tele-education.

Several strategies were adopted to address these difficulties. A traditional U-shape seating plan was adopted as it has the advantage of allowing all participants to see the monitor and data screen and it encourages dialogue with the presenter. A mud map (a seating plan for use by presenters in videoconferencing) was used to encourage interaction with the presenters. However, this seating plan was abandoned once it became obvious that it made no difference to the level of interaction.

Program structure

A formal structure was best suited to the technology and the size of the group. All sessions followed a similar format.

Case material was designed to ground the program in rural allied health workers’ experience. Discussion and group work encouraged networking among participants and over time helped build a sense of group. Consistent attendance helped achieve this goal so that, partly as a result, participants’ interaction with presenters improved in the second part of the program.

As the program progressed and presenters’ confidence with the technology increased, they moved away from the traditional lecture format and introduced more innovative strategies designed to maintain audience interest, such as the use of toys when counselling children affected by grief. As well, children’s art, diagrams, photos and video footage were used as teaching aids.

Information packages

Information packages were developed and forwarded to participants prior to the commencement of the program, and consisted of

The summaries proved to be an invaluable aid to participants in using the technology for learning; a detailed outline of the program actually appeared to assist participants in focusing on the information being presented. This acted as a safeguard for the program as it provided material to enable the session to continue if the equipment failed.

Site coordination

Site coordination involved the following tasks.

Also, using the following strategies, the site coordinator acted as a link between the presenter and the audience.

1. Presenters were given direct feedback on audience reaction to the information being presented by use of a mobile phone which was used to contact them during breaks, giving presenters the opportunity to clarify points and pick up on areas of interest to the audience.
2. Discussions were held with presenters between sessions to get their feedback for evaluation purposes and to fine-tune the next session.
3. The site coordinator consulted with presenters if and when a participant appeared distressed by the sensitive nature of the material being presented.
4. Between sessions, participants were contacted for their feedback. This was a component of the evaluation that was directly fed into the development of the program.

Project technology

Yellowlees and Kennedy (1997) provide a very clear definition of telemedecine equipment which applies equally to tele-education.

Telemedicine equipment

Tele-education equipment comprises three main components:

Depending on the type of system, other main components are the remote controls with the group system, and the keyboard and speakerphone with the desktop system.

Technical overview

Technical support was provided by Global Videoconferencing Technologies (GVT).

Project aim

To enable a lecturer in Melbourne to deliver a live presentation incorporating PowerPoint slides to and chair an interactive discussion with a tutorial group located in Bendigo.

Technical aim

To simultaneously deliver live video, audio and data streams.

Table 3 Children’s Mental Health Tele-education Program: The technical challenges

Technology as a tool–challenges and strategic responses

Although many successful examples of medical education programs using interactive video technology have been documented, integrating videoconferencing technology in the delivery of child mental health professional development programs poses a number of challenges to providers and receivers of the service. Before outlining the challenges, it is important to understand the clinical context that has informed the practice of child mental health and the environment within which the health system has operated.

Child psychiatry is an area of health care delivery, the foundation and success of which is characterised, strengthened and informed by the traditional therapeutic face-to-face relationship between therapist and client, that is, by the human interface. This is usually developed in the office of the therapist, where the therapist can easily pick up on both the verbal and non-verbal responses of the client. The challenge in using technology as an education tool in child mental health is centred on developing strategies to maintain the beneficial effects of the human interface.

The following table details the main challenges and strategies used to maintain the human interface.

Table 4 Interactive videoconferencing as an educational tool

Project findings

Methodology: qualitative data

The data contained in this section were collected primarily through interviews conducted by the rural site coordinator. Interviews were conducted with

Additionally, two focus groups were conducted after program completion to gather data on the value of program. The focus groups included managers and staff. Interviews and focus groups were designed to collect data on the following areas:

Presenter interviews were primarily designed to capture the experience of using this medium as a teaching tool.

The possibility of collecting independent quantitative data through referral patterns was explored but data from this source were not specific enough to assist in the evaluation of the program.

Attendance

The major reasons given by both participants and agencies were

These findings are consistent with the participant ratings, which can be found in the section ‘Content of Workshop, Evaluation Report’.

The program required a large time commitment from agencies and workers. Workers and supervisors reported that it would have been difficult to attend this program if it had been offered in a metropolitan centre.

The majority of workers attended the majority of sessions (see Independent Evaluation Report Attendance, Tables 1, 2, and 3). The most frequently cited reasons for non-attendance related to a lack of backfill. Other significant reasons for non-attendance included emergencies or other unplanned events requiring workers to miss a session. The program was offered over several months and in that time some workers took annual leave, others missed sessions due to illness and one went on a honeymoon (see Independent Evaluation Report Attendance, Table 4).

When interviewed, participants described some of the personal decisions they had made in order to attend.

The extent to which the education/information was taken back to agencies was also an indicator of the success of the program. From the responses to this question it would appear that informal dissemination of the information was good. In making the following comments, supervisors were reflecting on the impact that the program had on their teams.

Supervisors mentioned

Respondents mentioned

Participants indicated that they made material available from the program available to other workers at their agencies.

Effects on participants

Eleven of sixteen respondents reported gains in knowledge. Of the eleven workers who reported knowledge gains, six specifically mentioned the significance of working with children within the context of a developmental framework. One worker described her frustration in counselling a child whom she believed to be ‘stuck’. In applying a developmental framework to the problem she and the child were working on, she realised that the interventions she was using were not developmentally appropriate.

There was some feedback that the program content did not always meet the needs of the very experienced worker or the inexperienced worker. One very experienced respondent reported that she would have liked more indepth training.

Another respondent, who could be described as new to the field, found the case examples difficult and did not have the range of cases to apply content to.

These data are validated by the finding in the Evaluation Report that, overall, the sessions addressed participants’ areas of interest (see Independent Evaluation Report Content of Workshop, Figures Tele-education 6, 7, 8, 9,10 and 11).

Participants commented on the value of the program in validating their practice.

The focus groups identified that workers, because they were now better informed, were more confident in working with children with these problems.

Eleven participants reported a change in their perspective and practice in working with children as a result of attending the program.

Some comments by supervisors.

These comments support the finding in the Evaluation Report of a high level of anticipated changes to practice as a result of attending the sessions (see Independent Evaluation Report, Changes to Practice, Figures 14 and 15)

Response to video technology

Several respondents made the point that ‘Nothing replaces the person in the room’. Despite this, the response to the technology was positive and improved as the program progressed.

This corresponds with the findings from most respondents, who reported increased comfort with the technology as the program progressed. (see Independent Evaluation Report, How Comfortable was R with Tele-education, figures 17, 18 and 19).

However, presenters’ perspective about the levels of interactivity not being as high as they could have been were matched by comments made by participants in the individual interviews and focus groups.

Transmission problems also affected acceptance of the technology as a teaching tool.

The quantitative data show that levels of comfort with the technology were high, but of those participants who were uncomfortable with the technology at first, few remained so for all sessions.

Strategies to support the technology

Video technology is a tool. To be an effective education tool it must be supported by other strategies. The strategies used in the program were group work, discussion of cases based on rural practice and resource material.

Opportunities to break up into small groups between talks was really valuable.

Eight respondents mentioned the resource material.

Networking opportunities

From the response of participants it is clear that enhanced networking was an outcome of the program and strategies such as group work may have assisted in this.

Supervisors endorsed this point of view.

Presenter’s perspective

Relevance was a major issue for presenters. Presenters aimed to make the Children’s Mental Health Tele-education Program ‘useful and practical’. This involved two significant challenges. The first was that the program had to be sensitive to the issues faced by rural workers in child and family services: ‘Translating mental health issues into rural areas’, as one presenter put it. The second challenge was to ‘Think about the work that we do and make it specific to the situation of these workers’.

Understanding the education exchange as mediated by the technology and the implications of this was as important to presenters as it was for participants. Key components of this exchange–the presenter’s engagement with the audience, the audience’s sense of their part in the exchange–changed because of the medium. One presenter commented that she could see that the audience was ‘highly interactive at the other end and that lots of communication was happening’. However, this level of interaction did not flow through to the presenter on the monitor.

The sensitive nature of some of the material used in the program was a significant issue for presenters in regard to the possible impact of such material on participants.

This placed greater emphasis on the partnership between site coordinator and presenters. It then became part of the site coordinator’s role to follow up if any participant appeared to be distressed by session content. As one presenter said, concerns about resonance were partly resolved having the site coordinator ‘watching the audience and able to follow up on my behalf’.

An issue mentioned by all presenters and related to the above, was the difficulty of gaining a sense of the audience. In order to improve this situation, constant contact between presenters and site coordinator was maintained. While this did not completely address presenters’ problems in gaining a sense of the audience, it did prove useful in ensuring that the content and delivery remained on target with audience needs.

The tables demonstrate that tele-education is cost-effective for agencies and workers. When costs of technical support, site coordination, etc., are also considered, the savings

for presenters are not significant. While the basic costing does not suggest significant savings in using tele-education, when access and availability are also considered in this instance tele-education proved to be a cost effective strategy.

Costs

Table 5 Estimate of worker costs

Table 6 Estimate of presenter costs

* Worker and presenter costs are based on an average public sector rate.

** Travel costs are calculated using the public sector rate.

*** Worker costs include travel time to Melbourne/Bendigo.

**** Presenter costs include travel time to Bendigo.

The presenters involved could not have offered such an extensive program if travel had been required. From the evaluation it appears unlikely that participants would have been able to attend if travel to a metropolitan centre had been required.

When access and availability are taken into consideration, in this instance tele-education proved to be a cost-effective strategy.

Conclusion

A model for tele-education

The Children’s Mental Health Tele-education Program has achieved a number of significant outcomes.

The success of this program was based on the development of a model that can be replicated in other rural areas.

In discussing telemedicine Watters says, ‘It cannot be overemphasised that simply buying the box won’t enable you to practise successful telemedicine.’ Based on the experience of this project, successful tele-education requires devising strategies that connect the audience to the presenters. These stratagies should aim to replicate, as closely as possible, the human interction that is an intrinsic part of good education.

Maintenance of this human interface in tele-education is the goal that informed the model described below.

Figure 1 The Tele-education Model

Consultation

The success of this program depended on consultation with the rural allied workers the program was targeted toward.

Communication

This occurs between all parties involved in the program.

Most of this communication occurred using teleconferencing, email, telephone and fax.

Direct support to participants

The site coordinator provides support by managing a wide variety of practical issues in the distant classroom as well as mentoring the audience in using the technology.

High-quality equipment

High-quality equipment is essential in tele-education programs to achieve user acceptance of the technology as an educational tool.

Future directions

Response to the program exceeded all expectations, indicating that there is a significant gap in training provided to rural allied health workers in child mental health. Data collected from the program indicate that rural allied health workers play a significant role in pathways to care and ongoing support of children with mental health disorders and their families. This is an area that requires further exploration. Finally, sustainable training and education strategies need to be developed to support rural allied health workers in the critical work they do with children and families.

References

Adam, J. & Walker, R. 2000, Trusting Relationships between Organisations, unpublished paper, La Trobe University, Melbourne.

Blaskett, B. 1996, Personal Connections: An evaluation of the Royal Children’s Hospital Mental Health Service Videoconferencing Project, Melbourne.

Crump, W. J., Caskey, J. W. & Ferell, B. G. 1998, ‘Effect of a remote facilitator on small group problem solving: Potential Uses of Two Way Video Technology in Decentralised Medical Education’, in Teaching and Learning in Medicine, Vol. 10, No. 3, pp. 172—77.

Heath, C., Luff, P. & Sellen, A. 1997, ‘Reconfiguring Media Space: Supporting Collaborative Work’, in Video Mediated Communication, K. E. Finn, A. J. Sellen & S. B. Wilbur (eds), Lawrence, Erlbaum Associates, New Jersey, pp. 323—49.

Das, T. K & Tang, B. 1998, ‘Between Trust and Control: Developing confidence in partner co-operation in alliances’, in Academy of Management Review, 23, pp. 491—592.

Finn, K. E., Sellen, A. J. & Wilbur S. B. (eds) 1997, Video-Mediated Communication, Lawrence, Erlbaum Associates, New Jersey.

Fraser. C. 2000, Report of the Eating Disorders Consultation in the Loddon Southern Mallee Region, Bendigo Health Care Group.

Jameson, D. G. et al. 1995, ‘Broadband telemedecine: teaching on the information superhighway’, in Journal of Telemedecine and Telecare, 1, pp. 111—16.

Kamien, M. 1998, ‘Staying or Leaving Rural Practice: 1996 Outcomes of Rural Doctors 1986 Intentions’, in Proceedings from the 5^th National Rural Health Conference, pp. 449—54.

Kriesler, M. S., Snider, B. A. & Kiernan, N. E 1997—98, ‘Using Distance Education to Educate and Empower Community Coalitions: A Case Study, in International Quarterly of Community Health Education, Vol. 17, No. 2, pp. 179—93.

McCardle, G. E. H. 1993, Delivering Effective Training Sessions, Crisp Publications.

Commonwealth Department of Health and Aged Care 1999, Mental Health Promotion and Prevention National Action Plan: Under the Second National Mental Health Plan: 1998-2003, January, Canberra.

Mitchell, J. 1999, Evaluation Report on the first year of Bringing Child and Adolescent Mental Health Services to Rural Communities 1998—1999, John Mitchell and Associates.

National Rural Health Alliance 1999, Healthy Horizons: A Framework for Improving the Health of Rural, Regional and Remote Australians, A Joint Development of the National Rural Health Policy and the National Rural Health Alliance for the Australian Health Ministers’ Conference.

Reid, M. & Solomon, S. 1992, Improving Australia’s Rural Health and Aged Care Services, National Health Strategy, Background Paper No.11, Commonwealth Department Health and Family Services, Canberra, September.

Sheppard, L. & Mackintosh, S. 1998, Technology in Education: What is appropriate for rural and remote allied health professionals?, in Australian Journal of Rural Health, Vol. 6, pp. 189—93.

Report of the Project for Rural Health Communications and Technologies 1996, Telehealth in Rural and Remote Australia, Monash University, Melbourne.

Department of Human Services 1998, Victoria’s Mental Health Service: The Framework for Service Delivery Child and Adolescent Services, Melbourne, May 1998.

Weber, J. R., & Lawlor, A. C. 1998, ‘Professional Nursing Series by Videoconferencing’, in Journal of Continuing Education in Nursing, Vol. 29, Number 4, July/August.

Wootton, R. 1996, ‘Telemedicine: A cautious welcome’, in British Medical Journal, Vol. 313, pp. 1375—8.

Yellowlees, P. & Kennedy, C. 1997, Protocols and Guidelines for Telemedecine Applications, Department of Psychiatry, University of Queensland, Brisbane.

[Table of Contents]