The Research-Practice IMPACT Relationship AND THE MYTH OF - PDF document

The Research-Practice ‘ IMPACT’ Relationship AND THE MYTH OF RESEARCH-BASED • There is a long history of debate about POLICYMAKING AND the role of research in relation to policymaking and practice. PRACTICE • It has frequently been argued that there is a ‘research-practice gap’, where Martyn Hammersley instead there should be continuity: linear, The Open University, UK dialogical, or interactive. • This ‘gap’ has led to complaints from both Seminar given at the Centre for Organizational Research, University of Roehampton, March 2014 sides. Complaints about policymakers and Complaints about research practitioners • Not closely enough focused on the concerns of policymakers or practitioners; • Closed-minded or set in their ways, and • Fails to produce findings at the time they are therefore resistant to new perspectives; needed; • Committed to the dominant ideology and • Generates conflicting and confusing evidence; unwilling even to consider radical challenges that research findings may imply; • Provides evidence that is at odds with what is well known to policymakers and practitioners, • Untrained in the capacity to understand and so that its validity seems weak; make use of research; • Produces conclusions that are inaccessible to • Lacking in the motivation required to seek out practitioners, for example because too research evidence, and to reflect on their elaborate and qualified, or jargon-ridden. decisions in light of it. Evidence-based Medicine The rise of ‘research-based’ The evidence-based medicine movement, from policymaking and practice the 1980s onwards (Pope 2003), required that: • Spread of the notion of evidence-based practice • Clinicians must access research evidence to other areas: education, social work, etc. This about ‘what works’, and use only what has been fitted with the ‘new public management’ that scientifically validated, rather than relying upon became influential in the 1990s, aimed at their own past experience or outdated training. making public sector professionals more • Funding must be directed into research aimed ‘transparently’ accountable (Pollitt 1990). at discovering ‘what works best’. This research • The promotion of ‘evidence-based should use randomised controlled trials (RCTs), policymaking’, for example the ‘behavioural and funds must be allocated for systematic insights team’ (the ‘nudge unit’) in the Cabinet reviews designed to synthesise findings from Office (Haynes et al 2012) advocating RCTs in multiple studies. many areas of Government policy. 1

Changes in the research ‘contract’ Pasteur’s quadrant • Looking at the history of research funding, we can see a shift from the assumption that research will be beneficial in unpredictable ways, and should not be subject to external strategic management, to the requirement that it be more closely directed towards bringing about practical results and This is roughly what Hodgkinson et al (2001) refer to, in the context of management, as ‘pragmatic science’. I note that evaluated in these terms (Guston 2000). many of the same debates about Mode 1 and Mode 2, and • ‘Strategic research’ (Irvine and Martin, various versions of Mode 3 that purportedly combine all the benefits of the other two, have been present in management 1984), Pascal’s quadrant (Stokes 1997) science for some time, as in other fields: Huff 2000, Starkey and • Mode A and Mode B (Gibbons et al 1994) Madan 2001, Grey 2001, Hatchuell 2001, etc. Strategic funding of science The investment model • Application of the investment model to • Over the past 30 years public sector research funding first arose with research expenditure has increasingly been funded directly by government departments conceptualised in economic terms. • However, there was an early attempt in the • More specifically, it has been viewed as 1950s to apply this model to the funding of ‘investment’ – with the implication that a university science (Polanyi 1948; McGucken ‘return’ should be expected and should be 1978). This was defeated, but later came to be demonstrable. applied, primarily as a result of the very large • University teaching and research are among amounts of funding required by some areas of those parts of the public sector where this natural science. Relevant background here is conceptualisation is especially strained, the the increasingly industrial and commercialised arts and sport would be other examples. character of much science (Ziman 2000). The impact metaphor The impact agenda • The analogy is, of course, a physical one, and • A central concept in application of the of a relatively simple kind. investment model to research has been the notion of ‘impact’. • The model is of one object (say, a billiard ball) hitting another. • Institutional requirements have been introduced that researchers work to maximise • So, research is conceived as coming into the impact of their research, through contact with policymaking or practice and dissemination and engagement activities. sending it in a predetermined direction, in much the way that a cue ball can do this to an • Also required is that these activities and their object ball in billiards. ‘impact’ be ‘evidenced’. • This conception of the relationship between • Certain methods, notably RCTs, have been research and practice is held not just by lay viewed as intrinsically ‘high impact’ because people but also by many researchers. they ‘demonstrate’ ‘what works’. 2

A literary example of the kind of But is maximising ‘impact’ always ‘impact’ many researchers desire desirable? • Carol Weiss long ago raised doubts about this. ‘This savage novel of the bestial conditions She argued that what is important is not so among the stockyards and slaughterhouses much ‘to increase the use of research, but to of Chicago in the early years of the twentieth improve the contribution that research makes to century is perhaps the most influential and the wisdom of social policy’ (Weiss 1979:431) harrowing of all Upton Sinclair’s writings. […] • The idea that maximising impact is always So great was the furore caused by the desirable relies on an Enlightenment publication of this novel that the food laws of assumption about the progressive role of the United States were changed within six scientific knowledge. Yet this was questioned months’. (Back cover blurb for The Jungle: even in the eighteenth century, for example by Sinclair 1906) Rousseau. Questioning the impact metaphor Myth of research-based practice • The cue ball is struck so as to hit the object The myth = that research can tell us what is the ball. The first ball thereby has directionality best policy or practice. and momentum: Is this true of research? Three reasons why this is misleading: • The object ball is stationary, and only moves a) Research cannot validate value as a result of direct and immediate contact by conclusions: the ambiguity of ‘what works’. the cue ball. Is this true of policymaking or b) It can only provide limited and fallible practice? evidence about the effects of particular policies • Contact between billiard balls operates under or practices; and this is not its main function. laws predicting repeatable patterns of action, c) Research evidence must always be combined given certain background conditions. Is this with local knowledge in professional judgments true of the relationship between research and about what is best in particular contexts. policymaking or practice? Consequences of the impact model • What will be the consequences of this model for academic research? Is it, as Brewer (2011) claims, a ‘sheep in wolf’s clothing’ that enables social science to demonstrate its value? • Will ‘demonstrating impact’ remain forever a paper exercise, more bureaucratic nonsense with which one must appear to comply so as to obtain research funding? Or will it force the kind of change that RCUK and ESRC propose: the goal of achieving impact being built into the design of all research projects from the start (see Holmwood 2011)? 3