Researcher Education and Development Scholarship Conference

By Ros Attenborough (Evaluation Manager) and Anne Ploin (Community Knowledge Analyst), InFrame project, Universities of Glasgow, Edinburgh, and St Andrews.

This blog article describes a creative methodological journey through the evaluation challenges of a fixed-term research culture project

Why, and how, should we include evaluation in research culture projects? For those of us who know the human, research and societal cost of unhealthy, inequitable research cultures, we want to run projects that lead to positive, sustained culture change. We want to know what works, how it works, if it works in another context, and be able to explain and learn from unexpected or negative effects. When resourcing for research culture work is limited, we want to know how to focus our efforts – and so do our research institutions and funders.

The 2024 research culture initiatives in the UK (UKRI, 2024) report found that evaluation of research culture initiatives is often absent or minimal, and evaluation methods in this context need “strengthening and wider use”. This points to a gap between strategic need and everyday realities, as research culture evaluation challenges land with real people navigating uncertain circumstances in a Higher Education (HE) sector under pressure.

In this blog we invite you to join us in our specific evaluation challenge. Our context is the InFrame project: a complex two-year cross-institutional research culture project focused on collegiality and inclusivity in research leadership. Here, creativity, playfulness, flexibility, and collaboration enabled us to pivot – one step at a time – towards a robust evaluation approach when faced by common research culture evaluation challenges in HE.

Pivot 1: Evaluation strategy as collaborative and participative

In our project, Ros’ role held formal responsibilities for evaluation. However, research cultures being multiple and contested, a robust evaluation of our project’s culture work needed (many) others bringing in perspectives from across our team of frontline, experiential experts with their diverse skills and ideas. It didn’t work for one individual to develop and implement an evaluation strategy like a master plan, top-down, within a project that had to move pace from day one (like in this cartoon of Ros clutching her draft evaluation strategy and the ticking time-bomb of a project deadline).

Ros found that if she took on too much of evaluation by herself without letting others in, she risked becoming a bottleneck in the process. Moreover, even with relevant expertise, she was only bringing a single perspective and partial sightline to the process, missing the opportunity for bouncing ideas and critical dialogue. Collaboration made sense, and brought so much creative energy. Setting up this approach required wayfinding one step at a time: finding critical friends in the team to work with and then jointly building a more open, collaborative strategy. This is where Anne came in, with a complementary qualitative skillset but in a different role and based in a partner institution. Having an extra (and differently positioned) pair of hands enabled evaluation to cross project hierarchies and connect to the team in more varied and generative ways. Step by step, our project evaluation became a two-person collaboration at a strategic, methodological, and analytic level – and a full team, participative process at the level of data collection and interpretation.

Including more perspectives in evaluation planning can be win-win. In a pressured HE environment, collaboration means more sources of support, and can also lead to a more robust, participative process that carries legitimacy with more audiences. If you are evaluating from within a project or team, it’s worth noticing small opportunities to include more perspectives in your planning: a critical friend (or several) with a different positionality can make a crucial difference to the inclusivity and legitimacy of your approach.

Pivot 2: Creating spaces that feel different – the ‘evaluation cabin’

It was clear that everyone in our team had valuable and detailed insights into how our project was working. This is rich evaluation evidence, but it was initially hard to find the right settings for sharing these observations. We are a team of approximately 20 people in three institutions, with varied roles, expertise, and organisational positioning – most of us newly hired for the project, with a turnover related to fixed term contracts. To allow for collective reflection from inside our project, it was necessary to create a participative process through ‘spaces that feel different’: dedicated physical or virtual environments for different members of the team to share their insights in the form of evaluation stories. This began the design of an experimental evaluation event with facilitation experts in our team. We iterated in the months following, developing spaces for different participants – group or individual, in-person or online – and gave these spaces an identity: the ‘evaluation cabin’ (snapshots below).

What ingredients made these spaces work? We:

• changed the environment: booked a different space, or used a fun virtual background;
• used playful and informal visuals, objects and music: reminders of stepping outside the everyday and a vibe shift carrying permission to think creatively and critically;
• facilitated to increase psychological safety and participative decision-making: including check-ins and actively seeking minority voices, techniques from Lewis Deep Democracy;
• designed for power dynamics, positionality and confidentiality: created different spaces for different hierarchies or roles, because everyone’s viewpoint is affected by their position, and the social setting affects who speaks up and how;
• took stories seriously as evaluation data, outcomes and impacts of our project, and symptoms of the cultures we are working to understand; and
• offered material to respond to, such as outline reporting or narrative structures, to make evaluation more tangible and approachable.

Though it began as an experiment, the evaluation cabin became a central feature of our methodology and a way of valuing the insights and trust of cabin visitors. Unlike many evaluative methods, it puts participant relationships first and data-gathering needs second. Without taking relationships for granted, this values-led approach enabled us to build a rich catalogue of qualitative evidence with participants who wanted to be part of the process.

Pivot 3: Qualitative baselining, or benchmarking against the ‘origin story’

Evaluation of culture change often looks for post-initiative impacts. However, our two-year resourcing timeline ends not long after outcomes have begun. How could we capture impact under these conditions, and against what benchmark? Honouring the practical limitations of a finite project timeline, we decided to shift the focus of our evaluation to bring the history of our project into the frame: its initial context, the early design decisions and trade-offs, and the set-up and implementation phases.

By documenting our own starting point, we create a meaningful internal comparison point for evaluating change. Concretely, this looks like an evaluation reporting structure that includes an origin story, theory of change, and ‘what happened’ for each part of the project. By getting precise about the context of an initiative, we also make it more possible to extrapolate findings beyond our own context, because we’re clearer about what role our structure might have played in our outcomes. Properly documenting and analysing these elements creates a robust framework for comparison that is qualitative, sensitive to culture and context, and feasible within the boundaries of our project and evaluation.

Many possible evaluation strategies, no ‘correct’ answer – but some guiding principles

From the sheer variety of paths that multi-strand institutional research culture project evaluations take, we know that there aren’t simple or settled blanket solutions. Exchanging with colleagues in other institutions, we have seen projects take very different approaches: frameworks structured by deliverables, controlled trials, ethnographic and participant-observer approaches, and externally commissioned evaluation. This alone is evidence that we are navigating a relatively uncharted and varied landscape, underpinned by diverse assumptions and values for wayfinding.

In our own research culture evaluation journey, we found ourselves returning to four guiding principles:

1. A collaborative and participative approach, enabled by sensitivity to power dynamics.
2. A qualitative-led approach attuned to the dynamics of culture change work within change-resistant institutions.
3. A pluralistic mixed-methods approach, appropriate for teams of diverse roles and backgrounds that conceive of research culture in different ways.
4. A resource-flexible, opportunistic, and joyful approach – to sustain the work through sizable challenges and high stakes! See our joy-oriented Teams backgrounds for digital evaluation cabins below.

To conclude, we would like to highlight our final principle, which is about responding with creativity to the heavy, time-pressured challenges of research culture evaluation in our sector. For us, introducing joy, lightness and informality has allowed us to find a path to quality and robustness. This is not incidental to the quality of the evaluation we are creating: it is part of the process! We hope you might find something useful in these pivots for your own evaluation wayfinding.

Postscript: This is a work-in-progress, originally shared at the REDS conference in January 2026, developed into a public webinar, ‘Wayfinding for real-world, flexible evaluation of research culture projects’, and presented with emerging findings in July 2026. Our final evaluation reporting for InFrame will be complete towards the end of 2026. We welcome collaborative conversations and feedback on this form, especially if you are travelling along a parallel path in the landscape of research culture evaluation.

—

Acknowledgements:  We are grateful to the REDS blog and conference team, including Kay Guccione and two reviewers for their insightful and helpful comments. Our work is made possible by conversation partners working on research culture evaluation across the HE sector, visitors to the evaluation cabin, colleagues in InFrame and Culture Catalyst projects, and project funding from Wellcome.