Terrorism studies and the struggle for primary data
If research can only be as good as the data on which it is based, then the study of terrorism has been in serious trouble for decades, writes Bart Schuurman.
From the 1980s to the present-day, many of the field’s most influential scholars have lamented that a majority of articles rely on literature-review based methodologies and do not present any new, first-hand insights. Instead of authors talking with (former) terrorists, perusing the archives of counterterrorism agencies, administering psychological tests to extremists, or engaging in fieldwork, they have mostly resorted to talking amongst themselves. As a result, much of what we know about terrorists and terrorism has been derived from media sources – sometimes brilliant accounts that provide fascinating insights, but often short newspaper pieces of questionable accuracy and limited level of detail. Moreover, the field has put forward numerous explanations and theories of varying degrees of complexity, but has been unable to reliably assess their validity because the first-hand insights necessary to do so are few and far between.
Such arguments about the shaky empirical foundations of the field will be familiar to any student of terrorism who has had more than a passing interest in the subject. But that is perhaps one of the reasons why these issues have persisted for so long. Research on terrorism can be dated as far back as the 1960s, but came into particular prominence after the 9/11 attacks. These events, and the ‘war on terror’ that followed them, ensured that terrorism became a semi-permanent fixture within Western politics, society and media. This massive spike in attention for the subject, as well as the new sources of funding that came with it, brought many new scholars to the field and ensured a tremendous increase in output. Unfortunately, as authors like Andrew Silke and Avishag Gordon have shown, newcomers’ interest in terrorism was often short-lived; up to 83% of articles published in the 2000s were by one-time contributors. While some of these were of high quality, many lacked the in-depth knowledge of the field to truly help move it forward, especially where it came to issues like resolving the long-acknowledged scarcity of primary data.
Unfortunately, a relatively small group of dedicated scholars was not the only obstacle to a timely resolution of the primary-data problem. The long and short of it is that terrorism is in many instances simply a difficult subject to gather primary data on. One issue here is that many Western states have been fortunate enough not to suffer from a large number of terrorist attacks. This also means that terrorism is often a relatively ‘small-N’ problem, which increases data collection challenges. Then there are practicalities like a) finding (former) terrorists who could, for instance, be interviewed, b) convincing them to cooperate and c) making the time necessary to take these steps within research projects that are often conservatively budgeted in time and resources. Moreover, while it’s been argued that the potential dangers of engaging directly with (former) terrorists can often be managed, personal safety is still an issue that must be considered, as are the ethical guidelines for working with what are often vulnerable individuals and groups.
Concerns over the field’s apparent inability to gather more primary data, despite the problem having been known for decades, arguably peaked in 2014 when Marc Sageman’s provocatively argued that research on terrorism had ‘stagnated’. At first glance, such a damning statement from one of the most influential terrorism researchers in the post-9/11 period supports continued pessimism about the field. Yet, there was also considerable push-back; other leading scholars were quick to qualify Sageman’s piece as overly critical, and pointed to areas of real progress in the study of terrorism. Since then, both established and up-and-coming researchers have begun proposing that, actually, the primary-data issue may no longer be quite so problematic as it was. The question, however, is what the data says on the issue. Is ongoing pessimism warranted or has there been a change for the better? With the last in-depth assessment of the field dating back to Silke’s groundbreaking work in the early to mid 2000s, a fresh look was needed.
Over a two-year period, and with help from several research assistants and interns, I collected data on the circa 3.500 articles that had been published between 2007 and 2016 in the field’s nine leading journals. Like the work by Silke that had inspired this effort, I was principally interested in whether these articles used any kind of primary data, whether they utilized statistical analyses, and what their method of data collection was. Unfortunately, the number of articles with a specific ‘data’ (or even a separate ‘methodology’!) section was relatively small, meaning that in most cases my team and I had to trawl through the notes to ascertain what type of information had been utilized. After two years of this kind of masochism, we were happy to draw the analysis to a close with a decade of research scrutinized. The results have since been published in Terrorism and Political Violence and are available for early view.
The results strongly suggest that the study of terrorism has made significant strides towards finally addressing the shortage of primary-sources based research that has plagued it for so long. Not only did the study indicate a slow but steady upward trend in the percentage of articles using such data, but by 2016 a slim majority of all pieces published used some kind of primary data. Of course, there are qualifications; the inclusion threshold for counting something as primary data was purposefully set rather low. As a result, both years of field work and a single five-minute telephone interview suffice to tick the primary-data box. Still, I’d argue that the overall picture warrants considerable optimism.
Unfortunately, the field has not made similar progress in other areas. Qualitative research designs still reign supreme, with little growth in terms of the percentage of authors using statistical analyses. Moreover, some forms of data collection are still virtually nonexistent, such as experimental ones or clinical assessments of terrorists’ mental health. Most problematic, however, appears to be the enduring problem of one-time contributors and a lack of collaboration. Some 75% of all articles published between 2007 and 2016 are the work of single authors. Such limited collaboration cannot but constrain the scope of the research questions being tackled, as larger teams will generally have more resources in terms of time, finances and intellectual capital. Similarly, some 75% of all publications were the work of one-time contributors. While lower than the 83% noted by Silke in the 2000s, the relatively small group of dedicated researchers arguably continues to limit the speed and extent to which the field will actually be able to move forward on key issues.
The significant steps taken towards addressing the scarcity of primary data signal the maturation of terrorism studies as a field of research. Still, there is a lot of work to be done. Not only must we ensure this positive trend is perpetuated, but the quality and quantity of primary data being used still frequently leaves considerable room for improvement. No less important is that we continue to build a larger core of dedicated terrorism researchers, and that collaborative efforts become more frequent. There are good reasons to be positive about the prospects of terrorism studies as a field of academic inquiry. But there’s no room for complacency.
Welcome to the “Right Now!” blog where you will find commentary, analysis and reflection by C-REX’s researchers and affiliates on topics related to contemporary far right politics, including party politics, subcultural trends, militancy, violence, and terrorism.
“Right Now!” also provides a platform for republishing op-eds by our core team of experts (with due acknowledgement of course) which have been published by newspapers and on other blogs in order to further highlight the breadth of our work here at C-REX.