Giving value?
Diane Amans is a community dance consultant and director of Freedom in Dance, a Stockport based organisation. She is currently involved in training and mentoring dancers, managing projects and researching evaluation methodology. Here she asks some key questions about evaluating community dance
Evaluation, as I understand it, is working out the value of something -'making judgements, based on evidence, about the value and quality of a project' (1). As community dance practitioners we are expected to understand the impact of our work on the participants and to be able to demonstrate the extent to which aims and objectives have been achieved. In reflecting on the evaluation process in the context of dance work with disabled people, I revisited some concerns I have about power relationships and the validity of evaluation processes. There are four key questions that I use as a framework to evaluate my work and that of other community dance professionals - Why? What? Who? and How?
Let's start with WHY we evaluate:
- so we can be better practitioners? Or performers?
- so we can use evaluation in our planning?
- so we can check out the extent to which we've achieved our objectives?
- to measure ourselves against quality standards / targets?
- to demonstrate to funders and other stakeholders that we're delivering what we said we'd deliver?
- to get evidence for more funding?
Maybe we evaluate for all the above reasons but we need to be clear about why we are evaluating so we can decide on the evaluation methodology. Evaluation is part of our professional practice - the process of evaluation makes me engage in the debate about what constitutes good practice. But who decides what's good practice and what criteria are being used? In the context of projects involving disabled people - where are they in the evaluation process?
WHAT are we evaluating?
- the impact on participants?
- the extent to which we achieve our objectives?
Good practice in evaluation requires us to be specific about what we are measuring. But do we want SMART objectives in community dance? (Specific, Measurable, Achievable, Realistic, Timed) How specific can your objectives be when you have not met your participants? If we are going to set specific objectives for a group of people we have not met yet, how can we be sure that we are setting realistic objectives that the participants agree with? Can we avoid reinforcing a medical model of disability where professionals decide what is best for disabled people? We might think, 'No way would I do that'- but what do you say if you are a community dance practitioner who is asked to facilitate for a group which includes disabled people? Who decides on the aim & objectives? Where is the power?
If we are going to have SMART objectives they have to be measurable and some aspects of our work are easier to measure than others. We need to find a way to measure those aspects of our work that the participants and we find significant.
- Who decides on what is significant?
- What if the participants think one thing is significant and the facilitators find something else significant?
- Whose significant aspect is the one that counts?
WHO makes the judgements?
I have already raised questions about whose judgement counts, but WHO actually carries out evaluation of community dance projects involving disabled people?
- The Community Dance practitioner leading the work? (Is this a disabled person?)
- An outside pair of eyes? Is this a disabled person?
Does a non-disabled person facilitate the feedback session? Who reports on what has been said? There's a good deal of power here - like the power of the person taking the minutes in a meeting. The evaluator has this power. I've seen a non-disabled workshop leader ask for feedback following an integrated workshop. When someone made a positive comment his non-verbal responses invited further comments. When someone made a negative comment he acknowledged it briefly and moved on. The disabled participant later remarked -' I don't think he's really interested in us as individuals -just as a vehicle for making him look good.'
- Who decides the value of the work?
- Who makes these important value judgements?
- If it is you, what criteria do you use to decide what is good?
Methods of Evaluation
The WHY, WHAT and WHO questions all inform the methodology. HOW is the work evaluated? There are many ways of collecting evidence to help evaluate community dance projects - interviews, questionnaires, video, feedback discussions, photography, diaries, tape recordings, observation, 'graffiti walls', final performances. In choosing the appropriate methods for collecting evidence the evaluators need to consider, among other things, whether the methods are accessible and user-friendly for all the participants. These judgements require evaluators to have the necessary awareness and skills to ensure an inclusive approach.
If the participants do not use words to communicate, how can evaluators check whether their own beliefs about a project's value reflect the experience of the participants? I had a salutary experience when evaluating a dance project which included deaf teenagers. Through a signer I asked some carefully thought out open questions. In translating the questions the signer asked the participants 'Did you enjoy that?' (accompanied by an encouraging nod of the head!). I realised that I had to either learn to sign or take responsibility for briefing the signer properly. The timescales involved meant I chose the latter but the prevailing culture in which we worked meant that people were conditioned to give positive feedback. Ideally I should have found a translator who was experienced in a person-centred approach to giving and receiving feedback.
Making judgements
Each of these questions - why, what, who, and how - raises more questions. If we consider these in a practical sense now, how do we actually work out the value of community dance work? How much information or involvement do we need and what criteria are we using to make judgements about value? Are we going to look at aesthetics or the extent to which people are engaged in dance? Are we going to judge the quality of the relationships or look for evidence of participant enjoyment?
Clearly these questions cannot be answered without reference to the aims and objectives of the project - but the questions have to be asked before the work begins. If we do not think it through and cost it in before the project starts we may severely restrict our options when it comes to selecting methodology.
I use a range of different methods and I usually cost in an external evaluator. I feel this adds rigour to the evaluation process - and it keeps me on my toes. It is an invaluable source of professional development. In addition to discussing evidence for evaluation there is nothing quite like being asked to justify decisions and talking through alternatives with an experienced practitioner.
Conclusion
The questions and issues discussed above have relevance for all community dance work - not just the work which includes disabled people. However, when working with groups and individuals who probably have experience of service provision, which disempowers them, we need to make sure that our evaluation is inclusive. We do not want to reinforce an unequal power balance where community dance practitioners are just another bunch of interventionists who decide what's best for disabled people.
Diane Amans, community dance consultant and director of Freedom in Dance. Email: freedom@amans.fsnet.co.uk
(1) Woolf, Felicity: Partnerships for learning - a guide to evaluating arts education projects, ACE 1999.
The content of this site is proprietary to the Foundation for Community Dance and any access to this site
or the use of any content made by any person is expressly subject to these terms:
Unauthorised copying of any material (including artwork) on this site and the reproduction, storage,
transmission or the distribution of any content, either in whole or in part and in any medium or format,
without the prior written consent of the Foundation for Community Dance and, where appropriate, the author
or artist, is not permitted.
Please read our website terms & conditions by clicking here
Animated: Summer 2002