Evaluating Interprofessional Education

An important part of planning an Interprofessional Education initiative is deciding what to evaluate and how to conduct the evaluation.

An excellent resource is Freeth, D., Reeves, S., Koppel, I., Hammick, M., Barr, H. (2005) Evaluating Interprofessional Education: A Self-Help Guide Higher Education Academy. We recommend that you download and use this reference. We acknowledge and thank the authors for allowing us to borrow from their material below.

Another useful resource is Oandasan, I. and Reeves, S. (2005) Key elements of interprofessional education: Factors, processes and outcomes Journal of Interprofessional Care (May 2005) Supplement 1: 39-48.

First steps

How do you decide what you need or want to evaluate? Your interest could be wide-ranging or fairly narrow. Practical constraints of time and resources will have an impact. Before setting on a course of action, take some time to reflect on the following questions.

Who do I need to negotiate with?
- Who are the key stakeholders and what are their interests in this specific instance of interprofessional education?
- Who are the other evaluators and what are their interests and expertise?
What resources are available for the evaluation?
What is the allocated timeframe?
What is the purpose and scope of evaluation?

Determining the purposes of evaluation

Three of the most important axes of difference in relation to the aim and purpose of an evaluation are shown in the table below. (Freeth et al, 2005)

Some axes of evaluation

Developmental (formative) evaluation	Or And/Or	Summative impact assessment

Process focused	Or And/Or	Outcomes focused

For internal audiences	Or And/Or	For external audiences

Where should your evaluation lie on each of the axes in the table above? To guide your decisions you also need to ask yourself:

What is my priority and/or that of my organisation?
What do I feel confident to evaluate?
How much can I achieve in the time available to me?

Consider the format and objectives of the interprofessional education planned, and its wider context and role and answer the question: "What aspects of the interprofessional education do I want to find out more about?"

Freeth & Reeves (2004) elaborated on the work of Biggs (1994) to produce a 3P approach to evaluating IPE. According to their model, evaluation can centre on the PRESAGE (influences or constraints of the initiative), the PROCESS, or the PRODUCT. Determining which aspect is of most interest to you will help to determine the evaluation methods you might want to employ.

If you are interested in exploring PROCESS consider using qualitative methods such as observation. If you want to explore the PRODUCT or OUTCOMES of an IPE iniitiatve, a quantitative evaluation, for example using before-and-after questionnaires, might be more appropriate.

If OUTCOMES are your interest, the following chart might help you find a focus.

Classification of the possible outcomes of interprofessional education

1. Reaction	Learner's views on the learning experience and its interprofessional nature.

2a. Modifications of attitudes/perceptions	Changes in reciprocal attitudes or perceptions between participant groups. Changes in perception or attitude towards the value and/or use of team approaches to caring for a specific client group.

2b. Acquisition of knowledge/skills	Including knowledge and skills linked to interprofessional collaboration.

3. Behavioural change	Identifies individuals' transfer of interprofessional learning to their practice setting and changed professional practice.

4a. Change in organizational practice	Wider changes in the organizational and delivery of care.

4b. Benefits to patients/clients	Improvements in health or well-being of patients/clients.

Modified Kirkpatrick's Model of Educational Outcomes for IPE in Freeth et al. (2002, p.14).

The design of evaluation can vary depending on your purpose. Previous IPE evaluations have used a variety of designs including before-and-after, longitudinal, post-intervention, case study, randomized control trial and action research. In their review, Freeth et al. (2002) mention that the majority of published studies about IPE have used quantitative data with limited interpretation. They suggest that more interpretive, well-conducted qualitiative or mixed-mode studies would be beneficial to the field.

Validated measures include:

Heinemann, G.D., Schmitt, M.H., Farrell, M.P., & Brallier, S.A. (1999). Development of an Attitudes Toward Health Care Teams Scale. Evaluation & the Health Professions, 22, 123-142. (Note: For a copy of the scale and information on scoring, please contact Scott Reeves)
Hyer, K., Fairchild, S., Abraham, I., Mezey, M., Fulmer, T. (2000). Measuring attitudes related to interdisciplinary training: revisiting the Heinemann, Schmitt and Farrell 'attitudes toward health care teams' scale Journal of Interprofessional Care 14(3): 249-258.
Luecht RM, Madsen MK, Taugher MP, Petterson BJ. (1990). Assessing professional perceptions: design and validation of an Interdisciplinary Education Perception Scale. Journal of Allied Health 19(2):181-91.
Parsell, G., & Bligh, J. (1999). The development of a questionnaire to assess the readiness for health care students for interprofessional learning (RIPLS). Medical Education, 33, 95 – 100.
McFadyen, A.K., Webster V., Strachan K., Figgins E., Brown H., Mckechnie J. (2005). The Readiness for interprofessional learning scale: A possible more stable sub-scale model for the original version of RIPLS Journal of Interprofessional Care 19(6): 595 – 603.
Heinemann, G.D. & Zeiss, A.M. (Eds.). (2002). Team performance in health care: Assessment and development. New York: Kluwer Academic/Plenum Publishers. (Note: In this book the editors compiled and critiqued 66 instruments that measure facets of team performance.)
Mackay, Stuart (2004) The role perception questionnaire (RPQ): a tool for assessing undergraduate students' perceptions of the role of other professions Journal of Interprofessional Care 18(3):289-302.
Lindqvist S, Duncan A, Shepstone L, Watts F, Pearce S. (2005) Development of the 'Attitudes to Health Professionals Questionnaire' (AHPQ): a measure to assess interprofessional attitudes. Journal of Interprofessional Care 19(3):269-79

Ethical Clearanc

Don't forget that any research involving human subjects at the University of Toronto requires ethical clearance from the Ethics Review Office (ERO). Evaluations taking place in practice settings will also require approval from the appropriate hospital board.

Next steps

Educators at the University of Toronto are encouraged to evaluate any interprofessional education initiatives undertaken.

For assistance in planning the evaluation of University of Toronto IPE initiatives, please contact:

Scott Reeves, PhD MSc BSc PGCE,
Associate Professor, Department of Family & Community Medicine, University of Toronto Director of Research, Centre for Faculty Development at St Michael’s Hospital Scientist, Wilson Centre for Research in Education, University Health Network

[ Top ]

Home About Us Contact Us Centre Activities

Centre for Interprofessional Education, University of Toronto Copyright © 2006-2010	Last Updated: 12 August 2010