The arts do a great job of turning complexity into revelation, but do not do so well in evaluating their impact on the public. Francois Matarasso's landmark study on the social impact of participation in the arts, Use or Ornament (1997), noted from its questionnaire survey that about half of those interviewed (48%) reported feeling better or healthier since becoming involved in the arts. This was, however, loosely attributed by participants to feelings of improved well-being rather than a specific health benefit. The problem with well-being as an outcome of arts activity is that it too easily equates with well-meaning. It can appear woolly and congratulatory unless a guiding framework of evaluation is placed on the process of the activity itself. How can we better substantiate claims of improved well-being so that participants' testimony is not dismissed as anecdotal evidence?
The race to capture an evidence base to support the ascendancy of arts in health practice in recent years can sometimes seem like a treadmill. CAHHM's review for the Health Development Agency of documents from arts in health projects (Angus 2002) concluded that the majority of people working in this field appear to recognise that it is important to evaluate their activity. Many are attempting to do so, but they are struggling to find appropriate methods, and the evaluation they carry out is frequently inadequate. A widespread shortcoming is a failure to state clear aims for a project. To be able to seek the evidence of the effect of arts in health, it is first necessary to be clear about what effect is intended.
I think a useful pointer through this dilemma is something Michael Wilson said in his book Health Is For People (1975): "It is difficult to describe what we mean by well-being without asking the question: What is health for?" Perhaps we should do more to categorise both the intended and actual effects of an arts activity, e.g. better mobility, improved sociability, access to learning, change from adverse lifestyle. The sum total of such effects would constitute an increased capacity for living - and if that isn't health, I don't know what is. Health is expressed not only in the emotional and bodily engagement of participants in an arts activity, but also in the ancillary benefits that can be tracked from this. We can set a big goal like improved well-being but we need to prove small things first. This is not as obvious as it seems. Some years ago, I observed a dance project in London for people with obesity problems. The project's stated aim was to foster self-esteem and well-being in the participants, but it neglected to consider whether weight loss might be a measurable indicator of health gain from the activity.
The claim that participatory arts improve self-esteem also needs to toughen up. Self-esteem is not necessarily a good thing; some of us could do with less of it. A more pertinent watchword is dignity. What has always impressed me about successful dance projects with elderly people, for example, has been the inherent dignity of both the process and the participants' response. Dignity might be measurable too in that we all have an instinctive understanding of when our own has been violated. As the epidemiologist Jonathan Mann observed: "Injuries to individual and collective dignity may represent a hitherto unrecognised pathogenic force with a destructive capacity towards physical, mental and social well-being at least equal to that of viruses and bacteria." (Horton, 2003). The ability of the arts to help counter this is worth demonstrating. My local hospital trust's annual report, for example, states that one in six complaints received related to patients feeling they were not treated with dignity.
Another problem in arts in health evaluation is the tension between an instrumental approach that sees the arts activity as a tool to fulfil policy objectives and a transformational approach that trusts in the process itself to deliver outcomes. For example, a recent evaluation report on an arts in health project in Wythenshawe that had struggled to meet its aims laments that: "...we felt that what we perceived as an inward looking, therapy-oriented approach might militate against the potential of artistic processes that can create something that can be shared with others." (Storey 2004). A sense of working against the grain is a commonly felt frustration when evaluation is applied to arts in health.
Tom Smith's evaluation for CAHHM of the Common Knowledge arts programme in Tyne and Wear Health Action Zone (2003) highlights the need for the arts in health field to understand better the diversity of its practice and approaches as a precursor to setting aims. It focuses on learning outcomes rather than clinical outcomes and the Common Knowledge approach itself suggests it may be more useful to evaluate arts in health projects when they are grouped into programmes or portfolios rather than to just scrutinise them individually. Some projects may focus on the therapeutic benefits of the arts, some on environmental improvements to support health staff in delivering their care services, and others look at producing more creative kinds of health information. When doing arts in health work with communities, we may also focus on the concept of social capital where 'unity is health'. These are arts projects that start from the point of using creativity to enhance social relationships, reflecting growing evidence that good relationships are a major determinant of health. The point is that such diversity of practice requires distinctly different approaches to evaluation.
A study for CAHHM by Angela Everitt (2003) which observes five community-based arts in health projects has identified confidentiality issues in healthcare services as a barrier to validating evidence of the personal health impact from engaging in arts activity. So again we fall back on the voluntary testimony of participants themselves, which can be readily dismissed as 'soft'. But we should understand that in these early attempts to evaluate arts in health we embark on a process of discovery, not proof, and success is predicated on the quality of relationships built up between all involved in the project. It is not the arts activity alone that provides health gain - rather how it is delivered, and the environment and conversations around the activity, provide the intermediate indicators of perceived benefit. Which is why the artists, agency partners and participants can help by together tracking closely the evolution of a project to measure it against its objectives but also allow for the influence of befriending and unforeseen outcomes. To assist this process Everitt uses the methodology of logical framework planning.
A log frame has four columns and four rows. Please refer to the word icon at the top left of this page, entitled: Log Frame 1.
The first column, the narrative of the project, sets out the overall goal, the objectives which should ensure that the project travels in the direction of this goal, the activities that will help meet the objectives and the inputs or resources needed to conduct these activities. It has an 'if-then' logic, eg if these inputs are secured, then these activities will be undertaken, then these objectives will be met, then this goal becomes realisable.
The second column, indicators of effectiveness, addresses the question 'what would show us that we have been, and the extent to which we have been, successful in:
- getting nearer to realising our goal?
- going someway to achieving our objectives?
- undertaking our activities?
- securing the resources needed?'
The 'if-then' logic is continued both vertically and horizontally.
The third column, methods of verification, addresses the question ' how will we discover those things that would show us that we have been successful?' Again, the 'if-then' logic is pursued vertically and horizontally.
The fourth column, assumptions/risks, addresses those concerns that are summed up by the phrase 'but what if ...?' This column allows us to identify those factors that may affect the project pursing the programme as identified in the other three columns. This column helps to build realism into the project, to develop understanding of risks, and to identify factors critical to success. Some of these factors are outside of our control. Others alert us to the need to be vigilant or to introduce additional activities to address factors potentially detrimental to the project.
Log frames are useful for project planning and management generally as well as for evaluation design. For evaluation the second and third columns particularly help ensure that monitoring, review and evaluation are built into the project. Log frames also help to accommodate the sometimes different evaluation requirements of different stakeholders.
For a diagram that summarises the features of a log frame please refer to the word icon at the top left of this page, entitled: Log Frame 2.
If more arts in health projects would attempt to monitor and evaluate their progress along these lines it may be possible to build a critical mass of evidence allowing comparative study and statistical analysis to be undertaken. It could also break the current impasse on evaluation that I would summarise as follows:
Evidence can be supplied simply to show that art and health projects are addressing health and social participation, and this work can be described. But it is more difficult to provide evidence that these projects have an effect on health, social exclusion and civic participation. There is not a lot of reliable evidence on the effects of art and health projects, because it is not always clear what effects are intended. There is not much thorough evaluation either, but also it is not clear what would be acceptable as evidence. There appear to be some misunderstandings around the intentions and evidence for art and health activity. There seems to be a mismatch between the aims of the practitioners and the expectations of those requesting the evidence. The practitioners are addressing a wide range of particular circumstances in many ways and with a wide range of assumptions. Those requesting evidence seem to be expecting effects on individual health and behaviour, but they are not stating that explicitly. These conclusions were reached in a recent report by CAHHM to the Government's Social Exclusion Unit. (White, 2003)
In order to make progress in this search for evidence, it is essential that all parties clarify their intentions, assumptions and requirements. Practitioners need to state clearly what they are aiming to achieve. Funders and others requesting evidence need to state clearly for what effects they require evidence, and what would be acceptable as evidence. This could make us more attentive to a meaningful language of well-being as expressed by participants in the arts activity, and in the case of dance not just in verbal form but in the language of the body too.
Mike White is Director of Projects at Centre for Arts and Humanities in Health and Medicine, part of Durham University's multi-disciplinary School For Health. For contacts and more information contact
cahhm.info@durham.ac.uk or visit
www.dur.ac.uk/cahhm
References
Angus J. (2002). A Review of Evidence in Community-based Arts in Health. CAHHM, Durham
Everitt A. and Hamilton R. (2003) Arts, Health and Community. CAHHM, Durham
Horton R. (2003) Taking Dignity Seriously in Second Opinion. Granta
Mararasso F. (1997) Use or Ornament? - The Social Impact of the Arts. Comedia, Stroud
Smith T. (2003) Common Knowledge - An Evaluation of Sorts. CAHHM, Durham
Storey R. and Brown L. (2004) The Pathways Evaluation. LIME, Manchester
White M. and Angus J. (2003) A Literature Review of Arts and Adult Mental Health. Social Exclusion Unit
Wilson M. (1975) Health Is For People. Darton, Longman and Todd