ImPPPact: The three Ps of societal impact of research

Societal impact of research is a mature field with investigations going back more than half a century. Still, it remains an important topic both in science policy and in academic work, bolstered by policy demands for evaluating the societal impacts of research organisations. When we planned OSIRIS more than eight years ago, we wanted to move the field forward in different ways to address what we saw as biases in the mainstream literature.

The OSIRIS angle

First, we wanted to highlight the user side of impact. Our experience from studies of university-industry relations was that the bulk of this literature concerns what universities and their staff members are doing in such relations, even if their success rests heavily on activities in industry. Similarly, impact policies and investigations are primarily about what researchers and their institutions are doing – or perhaps should be doing – to generate effects beyond academia. We assumed that impact in many cases would be due to user activities more than the communication, interaction, and similar activities among researchers.

Second, we wanted to investigate the activities that happen before impact is clearly visible. A lot of the societal impact literature is directly or indirectly tied to evaluation. Proper evaluation requires identifiable outcomes that can be recognised as impact, and for some methodological approaches, also quantified. We sought to look at impact more as a process than a specific outcome, and to identify important activities that may happen before impact is clearly visible.

Third, some of our work has sought to cover what we saw as neglected types of research. This includes commissioned or contract research and the societal effects of non-academic research for example in industry or independent research institutes. Finally, we have in several analyses strived to use new methods and sources of data, including digital methods and sources and combinations of distinct data sets. In addition, the work in OSIRIS sought to scrutinise different contexts of impact, assuming that the user activities, types of research, and more, might vary between policymaking, industry, and healthcare.

This fruitful yet complicated setup has meant that OSIRIS has functioned more as a collective that in many distinct ways has sought to explore new angles in the mature field of impact investigations. In the last year, the team has nevertheless sought to tease out more general themes across the heterogeneous work. Many of our discussions have centred on three themes: preconditions, processes, and policies. These will be outlined briefly in the rest of this text, but they will also be discussed in more detail in later blogs.

Preconditions for impact

By preconditions we mean the characteristics that often need to be in place for societal impact to happen. These characteristics can be connected to the users and their organisations, the interaction with researchers, and the research itself. Preconditions is not meant as an absolute, but as a starting point for discussing important features that can be targeted by organisational practices and policies that strive for more or better use of research. Our work and discussions have centred on these characteristics in particular:

• Absorptive capacity: organisations that are regular and frequent users of research, seem to have a lot in common. They have a fair amount of personnel with a PhD degree and thus networks to research organisations, and they have high expectations and dedicated resources to the use of scientific evidence.
• Valuing science: even if it may not be sufficient by itself, our work indicates that the ones who use research extensively, also express very positive views about the intrinsic, and often also instrumental, value of science. For example, in our large survey among policymakers in Norwegian ministries and agencies, many of the respondents expressed ideals about science that are similar to what researchers themselves often state, such as publication in renowned scientific journals. This is an important source of legitimacy.
• Maturity and research mix: interviews and other data confirm themes from early investigations of impact. This concerns the maturity of the relevant research – it is difficult to put into use if aspects like trustworthiness, relevance, and reach can be questioned. It also refers to an observation that many active research users are in touch with several different types of research – from long-term knowledge development to close-to-consultancy activities.
• Practical need: especially in policy organisations, possibly also in other contexts, we find that research is much more easily (and probably faster) put into use when there is a clearly identified need and/or a concrete decision to be made at a specific time. Even if the research is potentially very relevant, it may more easily be ignored or put on hold if this is not the case.

The process of impact

Another theme at the heart of OSIRIS is how the process of impact generation unfolds. This can be elucidated in many ways, but our discussions have centred on three aspects. First, accumulation highlights how impact often builds upon long sequence of research activities rather than single projects. Networks between policymakers and scientists may in many cases stretch back decades. A practical implication is that connecting impact to a single project or publication – as is common in many evaluation approaches – conceals many of the important research and other activities that may have been necessary for the outcome.

Second, incrementality is a term that we use for an observation that many impact processes are not about implementing breakthroughs or radical new findings, but rather about making rather minor changes to an existing set of activities. Research is here a supporting activity, helping users to better understand why and how something works (or not) and how it can be scaled up, instead of being a unique source of new ideas. This also means that research can have an important impact that is almost invisible – if it helps stakeholders validate current practices and earlier decisions.

Third, many examples of impact are connected to major external events or crises. The COVID-19 pandemic is a good illustration, and we have earlier used it to discuss six major metaphors for or perspectives on how societal impact happens. As indicated by the two previous points, use of research is a mostly stable process, focused on accumulating relevant evidence and making small steps and changes. When major events occur, research is called upon to a much greater extent, and current reservoirs of knowledge are searched to find relevant solutions or to support or reject solutions that emerge from elsewhere. This seems to be a common way for undirected basic research to enter processes that lead to wider effects.

Policies that support impact

The question of policies has a double meaning, both referring to how science policy seeks to promote impact, e.g. by making it a formal criterion when evaluating research council proposals, and how the practices in all policy areas in different ways support use of research. Some of the work in OSIRIS has addressed the first, for example by investigating how impact is evaluated in funding applications and in broader research assessment contexts, and by investigating the effects of different types of R&D funding to firms. Effects seem to be limited except for support to firms that are relative newcomers to research activities, at least in the short and medium term. This has generated significant debate, highlighting the challenges in measuring impact.

Our large-scale survey of Norwegian ministries shows that there are major differences between policy areas in their use of research. This probably reflects long-lasting cultures and practices but also characteristics of the different policy areas and their varied research landscapes. We do not suggest that there is one best way of organising evidence-based policies, but that there is an untapped potential in learning across policy areas and in making this work more systematic and perhaps less incremental.

A major suggestion from the OSIRIS perspective and work is therefore that policies for impact need to take the user side much more seriously. How can research more explicitly contribute to the absorptive capacity of relevant users? Can science policy instruments – beyond than the selection of a few societal challenges – help make clearer formulations of needs and make these needs themselves more based on evidence? These are questions that we look forward to discussing with our key stakeholders in the months ahead.

This text is part of a series of blogs from the OSIRIS centre. We will publish a number of these blogs in the coming weeks leading up to the final conference that we organise together with R-QUEST (our sister centre focused on research quality) on November 8^th. To sign up for this conference, which will be held in Oslo but also streamed to a wider audience, click here.