Evaluating AOD Training and Programs

What is evaluation?

Evaluation is the systematic assessment of the process and/or outcomes of a project or program, compared to a set of explicit or implicit standards. The findings from an evaluation may be used to contribute to the improvement of the project or program. Evaluation has been defined as:

"The systematic assessment of the relevance, adequacy, progress, efficiency, effectiveness and impact of a course of action."

Evaluation is the process by which we decide the value or worth of something:

"Evaluation is a judgement... (involving) (1) observation and measurement... (2) comparison of what you observe with some criterion or standard of what you (or the group you represent) would consider an indication of good performance."

There are many models used for evaluation which tend to define key evaluation concepts in different ways. When determining which model is right for your organisation ensure attention is focused on the application of an evaluation framework which is practical, useful and intuitive.

Using evaluation techniques to address workforce development challenges

Evaluation is an important component of any workforce development intervention or program initiated in the workplace. Many principles and practicesmay be useful for the evaluation (and continuous improvement) of workforce development initiatives such as:

• Implementing a new performance appraisal system
• Conducting a clinical supervision or mentoring program
• Developing a stress management program
• Implementing a professional development program
• Engaging in an organisational change initiative.

Evaluation is conducted for three purposes:

1. Rendering judgments - to determine if the project/program has done what it set out to do (an accountability perspective)
2. Facilitating improvement - monitoring the project/program to identify what is working well and what is not in order to modify the approach where necessary
3. Knowledge generation - contributing an understanding of the area addressed by the project/program.

Evaluations need to be conducted systematically and rigorously, using appropriate methods of data collection which address clearly defined project/program:

• Processes - the actions and strategies employed
• Impacts - the shorter-term effects or changes which the project / program aims to achieve (specified in the objectives)
• Outcomes - the longer-term effects or changes related to the overall goal of the project / program.

Planning the evaluation

Four key steps in planning the evaluation are:

1. Establishing a reference group
2. Designing an evaluation plan
3. Deciding on the types of indicators to be used
4. Deciding on the data collection methods.