The community-engaged artist is often sponsored by a state or private bodies that seek knowledge about a community for its own ends and from their own vision, whether well-intentioned or not. What does it mean to understand a community on its own terms particularly when it comes to evaluating a project shaped with others? In so doing, how might we think of evaluation differently? How might we move away from practices that reiterate problematic forms of extraction, documenting and representation, and instead centre artists and communities as co-theorising subjects. What might an artist-led learning and evaluation process look like? How might this reshape how we think of ‘knowledge’ and ‘evidence’? What is ‘knowledge’ or ‘evidence’ in arts projects? And what can it be?
Image of Bigoa Chol as a part of Arts Gen’s Practicing Utopias publication to be released
Although resisting a universal definition evaluation often involves the description and judgement of a project, program, policy, etc. ‘Evaluation is the production of a systematic determination of merit, worth, or significance of an object through the application of criteria and standards based on relevant evidence.’
In practice, when applied to social/cultural/community/health programs, evaluation means assessing their effectiveness, essentially asking ‘Is it successful?’. Traditionally, this involves two types of questions and processes:
Evaluation relies of measuring the changes allegedly produced by the project. For that, it seeks to “gather data” to create concrete, observable and measurable indicators that help understanding, describe and assess how a project is ‘doing’. That’s why most people relate evaluation to ‘performance reviews’ and ‘satisfaction surveys’. In practice; it can involve a range of quantitative and qualitative methods to document a project’s process and outcome.
Arts impact evaluation frames often assume instrumental benefit of the arts: artistic outcomes are usually most valued when they demonstrate their contribution to non-artistic outcomes. Around community-engaged art projects, the contribution to social inclusion and wellbeing are often taken for granted. Generally, the exercise spins around designing the ‘measures’ to show a pre-defined expected change. However ‘measuring’ is a tricky and challenging exercise: cultural measurement is defined by an economistic logic of indicators (fuelled by funding needs in the arts) and the assumed neutrality and objectivity of numbers (boosted by our data-driven times).
The focus here is mostly technical and methodological with the main question being HOW to measure adequately? How to accurately represent cultural forms in data? How the (presumed) positive impacts of arts might be best captured?) There’s a range of technical issues regarding the difficulty of capturing relevant aspects and dynamics involved in community arts projects: e.g: tangible/intangible, process/result, individual/collective elements, and specific dynamics of different artforms.
However the shift in the question, from ‘how’ to measure to ‘why’ measure dramatically changes the conversation about arts evaluation. The latter highlights underlying issues about How we define and understand ‘VALUE’. Critical approaches to evaluation highlight that any discussion around ‘worth’ is essentially contested and political, and entangled in questions of power, authority, and influence. Arts evaluations carry philosophical and political challenges that open other specific and broad questions:
The key point is therefore that ‘value’ is not only ‘captured’ but CREATED in the evaluation process.
TASK Have a think about your experiences with evaluation, as a participant or as someone who might have had to administer. What were you asked to do? What did this say about what was considered important to the project? What did you feel might have been missed from the particularly evaluation? Write a short paragraph.
Sophia Chai reflecting on Evaluation as Practice
Critical approaches acknowledge that our views, ideas, and positions underlie any evaluation and research process. What counts as credible or relevant data is never “Neutral” when determining significance or worth. Changing the evaluation criteria and questions changes the whole understanding of a project’s value.
The tension between the technical demands and political-philosophical questions involved in arts evaluation is an inevitable part of it. BUT here’s a few approaches that offer meaningful ways on how to navigate it:
they bring communities and arts practice to the mix.
they care about respect and accountability when working with others
they promote transparency and openness to different modes of valuing and learning about projects
(i) Decolonising research (and evaluation!)
Perhaps the most important and inspiring reason to re-think evaluation frames is to recognise that knowledge-making and representation are not neutral but often shaped/framed from vantage point of predominant views. Goes beyond mere technical debates and puts existing methods/techniques in the service of communities’ own questions and practices: It does not dismiss Western methodological approaches, but engages them in meaningful and useful service to their values and people:
“It is about centring our concerns and world views and then coming to know and understand theory and research from our own perspective and for our own purposes” Linda Tuhiwai Smith
This highlights that evaluation can be more than data collection = “IT is action, it is process, it IS the seeking of meaning in community” (Jo-ann Archivald, 2019, Decolonising Research: Indigenous Storywork as Methodology).
(ii) Community-led evaluation
Evaluation projects can be framed as Community-engaged practice.Working with communities through arts is social, cultural, and political. Reflecting and assessing projects with participants, teams and communities involved is an important form of evaluation. Make the space and time for them to set the terms of the evaluation process. This often allows practical assessment of projects around: what worked well, what didn’t, and what would need to improve or change? This can lead to all kind of conversations, creative thinking and project shaping. Other attempts include giving space for participants/artists to lead the reflection process, define questions and evaluation criteria; contribute to information gathering, interpretation and creation can activate important things within projects
(iii) Research-Creation: Practice as evaluation
A way of doing theory/thinking that emerges from creative practice is to centre art practice to learn about art projects and using art as a primary tool. This approach attempts to understand/document/explore the aesthetic and embodied experiences within projects, instead of measuring a pre-defined impact:
This approach is a more experimental and open approach and in practice it might look like an art project in itself.
Image taken from Honesty Policy by Sirak Keeghan
EXAMPLE: In creating Honesty Policy, artist Sirak Keeghan used a series of creative workshops and public showings – part music jam, part open mic and performance dialogue to create a podcast that shares local stories of resilience, resourcefulness and created conversations about positive mental health. Read more and listen here.
[On the Politics and technical challenges of Arts Evaluation]
[On Decolonising Research Perspectives]
[On Theoretical views of Practice-as-Research?]