Thesis now published – Troublemakers in the Streets?

My thesis “Troublemakers in the Streets? A Framing Analysis of Newspaper Coverage of Protest in the UK 1992-2017” is available on the website of the University of Glasgow since last week. It scrutinises how mainstream news media in the United Kingdom have framed domestic protest over the last three decades. I will (try to) publish parts of this research for different audiences over the next year. But here I wanted to summarise a few key points.

The Puzzle

The motivation for this research is twofold. First, protest is really important for democracy. It is one of the few tools with which ordinary citizens can make their voices heard in the political arena quickly (as opposed to waiting for the next election) and directly (as opposed to calling a member of parliament and hoping they will champion an issue).

However, and this is the second motivation, it seems that most protests have a rather hard time of accomplishing anything. Their basic problem is that in order to sway public opinion about an issue, they need to be heard and understood. To be heard is not an easy goal to reach in the first place. Bystanders of a protest are only a tiny fraction of the audience protesters hope to speak to.

Hence, the main obstacle is the media. Yet most protests never actually make it into the news (e.g., Amenta et al. 2017; Wouters 2015). And when they do, they are often covered in a less than supportive way: the news highlight the disruption of normal life through protest, how odd protesters look or how young/old/over-/undereducated they are, and clashes erupting between protesters and police. This reporting then often crowds out descriptions and explanations of the cause and grievances voiced through protest. In other words, while reporting on protests, journalists focus on protesters and their methods over the message.

This phenomenon was dubbed the (journalistic) protest paradigm (Chan and Lee 1984; McLeod and Hertog 1999). Basically the idea is that even before the actual event happens, journalists have a world view that tells them “where to look (and where not to look), and informs them about what to discover” (Chan and Lee 1984, 187). This, many scholars argue, is the reason why reporting about protests is often similar, irrespective of the message and conduct of protesters.

So protesters basically have no chance to be understood, or do they? There is quite a bit of evidence supporting the protest paradigm claim already. Of the 52 studies I reviewed, 32 support the notion that the protest paradigm is the driving force behind reporting. As you can see in the table below, research in this field covers several decades and a variety of countries.

Table 2.1: Empirical Studies on the Protest Paradigm in Legacy Media
Study Country Case Period Support for PP Measured
McLeod and Hertog (1992) US 1986–1987 Yes Other
Smith, McCarthy, McPhail, and Augustyn (2001) US 1982, 1991 Yes Episodic vs. Thematic
McFarlane and Hay (2003) Australia 1999 Yes Marginalisation Devices
Boyle, McCluskey, Devanathan, Stein, and McLeod (2004) US 1960–1999 Partly Episodic vs. Thematic
Boyle, McCluskey, McLeod, and Stein (2005) US 1960–1999 Yes Valence
Brasted (2005) US 1968 Yes Marginalisation Devices
Dardis (2006a) US 2002–2003 Partly Marginalisation Devices, Valence
Dardis (2006b) UK, US 2002–2003 Partly Marginalisation Devices, Valence
Gavin (2007) UK 2000 Yes Marginalisation Devices
Jha (2007) US 1967–1999 Yes Episodic vs. Thematic
McLeod (2007) US 2006 No Marginalisation Devices
Boyle and Armstrong (2009) US 1960–2006 Yes Valence
Di Cicco (2010) US 1967–2007 Yes Other
Gavin (2010) UK 2008 Yes Marginalisation Devices
Edgerly, Toft, and Veden (2011) US 2006 Yes Marginalisation Devices
Harlow and Johnson (2011) US, Egypt 2011 Yes Marginalisation Devices
Sela-Shayovitz and Hasisi (2011) Israel 2000, 2007 Yes Other
Boyle, McLeod, and Armstrong (2012) International 2007–2009 Yes Valence
Corrigall-Brown and Wilkes (2012) Canada 1990 No Other
Weaver and Scacco (2012) US 2009–2010 Yes Marginalisation Devices
Xu (2013) US 2011–2012 Yes Marginalisation Devices
Young (2013) US 2011–2012 Yes Marginalisation Devices
Baysha (2014) US, Russia 2011 Yes Media Frames
Lee (2014) Hong Kong 2001–2012 Yes Marginalisation Devices
Ghobrial and Wilkins (2015) Egypt, Saudi Arabia, Tunisia, US 2011 Yes Other
Gottlieb (2015) US 2011–2014 Partly Media Frames
Hughes and Mellado (2015) Chile 1990–2011 Partly Marginalisation Devices
Spyridou (2015) Cyprus 2013 Yes Media Frames
Wouters (2015a) Belgium 2003–2010 Partly Other
Wouters (2015b) Belgium 2003–2010 Partly Episodic vs. Thematic
Coulter, Browne, Flynn, Hetherington, and Titley (2016) Ireland 2002–2003 Yes Valence
Oz (2016) Turkey 2013 Yes Media Frames
Power, Haynes, and Devereux (2016) Ireland 2014–2015 Yes Marginalisation Devices
Reul, Paulussen, Raeijmaekers, van der Steen, and Maeseele (2016) Belgium 2011 Yes Marginalisation Devices
Shahin, Zheng, Sturm, and Fadnis (2016) Brazil, China, India 2011–2013 Partly Marginalisation Devices
Veneti, Karadimitriou, and Poulakidakos (2016) Hong Kong 2014–2015 Partly Media Frames
Elmasry and el-Nawawy (2017) US 2014–2015 No Other
Kyriakidou, Osuna, and Hänska (2017) Spain, Greece, Germany 2011 No Marginalisation Devices
Kyriakidou and Olivas Osuna (2017) Spain, Greece 2011 No Marginalisation Devices
Leopold and Bell (2017) US 2014 Yes Marginalisation Devices
Trivundža and Brlek (2017a) Slovenia 2012–2013 Yes Marginalisation Devices
Ismail, Torosyan, and Tully (2019b) US 2014, 2017 Partly Other
Kilgo, Mourão, and Sylvie (2019) US 2013, 2014 Yes Marginalisation Devices, Media Frames
Mourão (2019) Brazil 2013 No Marginalisation Devices
Coombs, Lambert, Cassilo, and Humphries (2020) US 2016 Yes Media Frames
De Cillia and McCurdy (2020) Canada 2018–2019 No Marginalisation Devices
Gil-Lopez (2020) US 1998–2017 Yes Marginalisation Devices
Harlow (2020) US 2017 Yes Media Frames
Harlow, Kilgo, Salaverría, and Garc'ia-Perdomo (2020) Worldwide 2014 Partly Framing, Sourcing, Marginalizing Devices
Kim and Shahin (2020) Korea US 2016–2017 Partly Media Frames
Papaioannou (2020) Cyprus 2014 No Media Frames
Umamaheswar (2020) US 2016 Yes Marginalisation Devices

However, not all studies support the existence or importance of the protest paradigm. Furthermore, despite a wide range of studies, our knowledge about the topic still has several gaps:

  1. The design of most case studies is rather narrow, as they analyse single, often radical, protest events or scrutinise the coverage surrounding a specific issue or movement.
  2. There is a lack of longitudinal studies, which might be a problem as the media landscape has changed considerably over the last decades.
  3. There is a wide range of operationalisations of the protest paradigm, as studies code marginalisation devices, valence or framing.
  4. Especially in earlier studies, there is often no quantification of articles falling into one of several categories of reporting.
  5. The literature offers no clear theoretical framework for interpreting findings.

The last two gaps become apparent when comparing two of the studies from the table above: McLeod (2007) and Edgerly, Toft, and Veden (2011). Despite studying the same case, using almost the same data and the same method, they arrive at different conclusions. While McLeod (2007) asserts that expectations derived from the protest paradigm were not met in the news media coverage, Edgerly, Toft, and Veden (2011) describe a different finding:

“the protest paradigm continued to be a powerful organizing principle in media coverage of the protests. Most significantly, organizers did not generate comprehensive coverage of their legislative goals in the mainstream press, or overcome the episodic and tactical framing of most reporting on political protest” (p. 329).

I tried to fill the above-mentioned gaps through my case study design, by suggesting a theoretical framework for the interpretation of results, by developing a new method and by providing empirical findings through a two-step analysis. These are introduced briefly below.

The Case

Instead of focusing on specific protest events or movements, I included all domestic protests in my research — as far as they had been covered by the media. During the last decades the driving force behind many changes in protest organisations, the media and the wider society was arguably the Internet. So I chose the year 1992 as the starting point in which these changes haven’t yet taken place and collected data until the end of 2017. I chose the United Kingdom for several reasons but most importantly I expected that the unique divide between left and right wing, as well as broadsheet and tabloid news, would be an advantage for the study. Several scholars have suggested that these categories of news outlets will be important for how protest is covered. I have a whole chapter on how I constructed the novel dataset of newspaper data for this study, but here it will suffice to say that it included 27,496 newspaper articles from 8 different outlets (and that I discarded 95% of the initial sample during cleaning).

The Theory

I chose to trust the framing concept as the theoretical backbone of my research. The advantage of framing is that it connects research about media content with research on media effects. Although I don’t study media effects, findings from this literature provide guidance on how to interpret empirical results about media content. See, studies like McLeod and Hertog (1999) were still under the impression of early framing effects research which suggested that framing could be used to strongly shape peoples’ opinions about issues. Hence researchers studying the protest paradigm usually expected it to delegitimise protest in the audience’s minds and render it ineffective.

However, more recent research expects that such strong effects will only occur if one frame completely dominates the discussion. Under normal circumstances, namely when multiple interpretations are available, individuals tend to deliberate and personal values and evaluations of the quality of arguments become more important (e.g., Chong and Druckman 2007a, 2007b). This means that the emphasis on the sheer presence of the protest paradigm might have led to overemphasising the importance of the phenomenon. As long as alternative frames are available, the protest paradigm might not do much harm. Yet to find how dominant the paradigm is, I had to assess how prevalent or even dominating certain frames are in the discussion. In other words, I had to quantify the usage of frames. This led me to the first two research questions guiding my thesis:

RQ1: How do British newspapers frame the coverage of domestic protest events?

RQ2: How — if at all — did the framing of protest reporting change over the last 26 years?

As a second step, I wanted to know what triggers coverage following the protest paradigm. The third research question was consequently:

RQ3: Which factors explain the choice of frames by the news media when covering domestic protest events?

I took hints from several excellent studies (e.g., Lee 2014; Kilgo and Harlow 2019; Wasow 2020) and combined the factors which I expected to shape coverage about protest. The table below shows the hypotheses I worked with.

Table 2: Hypotheses
H1 Stories on protest events mainly use delegitimising framing as described in the protest paradigm literature.
H2a Delegitimising framing decreases in salience over time.
H2b Legitimising framing increases in salience over time.
H3a The goal of a protest determines how likely delegitimising framing is to be used, following a hierarchy of social struggle that is ordered from highest to lowest probability: political, anti-war, labour, police, and social-issue protests.
H3b The goal of a protest determines how likely legitimising framing is to be used, following a hierarchy of social struggle that is ordered from highest to lowest probability: social-issue, police, labour, anti-war and political protests.
H3c When protesters break laws or when they engage in violence, delegitimising framing is used more often.
H3d When protesters are peaceful and obey the law, legitimising framing is used more often.
H3e When the state uses repressive tactics on peaceful protesters, legitimising framing is used more often.
H3f Right media outlets use delegitimising framing more often than left media outlets.
H3g Left media outlets use legitimising framing more often than right media outlets.
H3h When there is a divide between the ideology of the protest’s goal and the ideological leaning of the outlet, delegitimising framing is used more often.
H3i When there is agreement between the ideology of the protest’s goal and the ideological leaning of the outlet, legitimising framing is used more often.
H3j Tabloid media outlets use delegitimising framing more often than Broadsheet outlets.
H3k Broadsheet media outlets use legitimising framing more often than Tabloid media outlets.
H3l At the beginning of protest coverage of an event, news will be event-driven and hence are more likely to contain delegitimising framing.
H3m The more time passes between the start of a protest event and publication of an article about it, the more likely it is that the article contains legitimising coverage.

The Method

To measure if the protest paradigm was the driving force behind protest coverage, I measured framing. I did so using a new method. I know that a surplus of different ways to operationalise theory is not solved by adding yet another way to measure a concept. Yet, I did so anyway for two reasons: firstly, it wasn’t always clear to me how previous studies coded content and I wanted to be completely transparent about my process of how I arrived at a particular set of frames. And secondly, the dataset I had amassed for my case was too large to analyse it with purely manual content analysis.

The flow chart below shows the different steps of the new method:

Figure 4.1: Frame Analysis Method

Obviously, the steps are explained in detail in the thesis, but just to highlight two key aspects: the understanding of frames here is that they are latent variables organising the content within a story. Instead of trying to measure these variables directly, it is both valid and more reliable to measure individual, more explicit parts before combining them into frames. I followed Matthes and Kohring (2008) who coded frame elements based on the seminal definition by Entman:

“To frame is to select some aspects of a perceived reality and make them more salient in a communicating text, in such a way as to promote a particular problem definition, causal interpretation, moral evaluation, and/or treatment recommendation for the item described” (Entman, 1993, original emphasis).

In Step 4 of the method, I use factor analysis to detect the underlying latent dimensions and interpret them as frames.

The second key element of the method is that after manually coding a sample of the data, identifying the main frames, and determining the presence or absence of each frame in each manually assessed article, I use Automated Content Analysis (ACA) to code frames in the remaining dataset.

Framing: How?

As said above, I proceeded with the analysis in two steps. The first one was to determine how UK newspapers frame protest. I identified seven frames, which you can see as factor loadings below (click on the image to enlarge it).

Figure 6.7: Factor Loadings for Selected Frames

I hope you agree that this visualisation makes it quite easy to interpret the factors as frames. Basically, high values mean that a code is important for a frame. When you look at the Troublemakers frame, you can see that the main topic discussed in this frame is “Violence/Crime”; the biggest risks identified in stories using this frame are “property destruction” and a danger for public safety; protesters are blamed for this risk. In other words, the Troublemakers frame does exactly what the protest paradigm literature predicts. And so do three other frames I found. I sorted the frames into two categories, delegitimising and legitimising, to discuss expectations more easily. The two legitimising frames defy expectations from the protest paradigm literature: the Cause & Grievances frame explicitly highlights why protesters took to the streets and support the protesters’ assessment that their grievance poses a risk that should be addressed. A more thorough discussion of these results can be found in Section 6.2.4 of the thesis.

After determining the presence or absence of these frames in the sample of manually coded articles, I trained a number of machine learning algorithms with different pre-processing chains and evaluated their performance. The results shown in the table below are encouraging, to say the least.

Table 6.11: Classification Performance of Frames (Top Models)1

Frame Accuracy Precision Recall F1 Algorithm Preprocessing
Police Violence 0.885 0.892 0.943 0.917 textmodel_nb P-L-I-T-3
Law & Order 0.885 0.882 1.000 0.938 textmodel_svm P-L-S-W-I-T-3
Cause & Grievances 0.769 0.789 0.882 0.833 textmodel_svm S
Righteous Struggle 0.885 0.880 1.000 0.936 textmodel_svm I-T-3
Troublemakers 0.827 0.848 0.951 0.897 maxent N-W-I
Decay of Morals 0.904 0.915 0.977 0.945 maxent N-W-I-T-3
Nuisance 0.904 0.898 1.000 0.946 maxent P-S-I

Using the models which emerged as the best ones, I coded the remaining data. What is important to note here is that I treated frames as independent from each other. Each article can contain one, several or none of the frames.

Figure 6.10: Percent of Total Articles per Year Containing a Frame

Looking at the figure above, two things become clear quickly: firstly, none of the frames and neither of the two frame categories dominate the discussion entirely at any point in time. While the delegitimising frames are in the majority overall, there is a strong presence of legitimising frames. And secondly, there is not that much change in framing overall over time. Most fames end up at almost the exact level they started, even though some fluctuate quite wildly.

Of course, I didn’t just eyeball this result but confirmed it through a simple OLS regression. Specifically, I calculated one OLS model for each frame, using the year as independent variable (starting with 1992 as year 0) and the percentage of total articles per year containing a frame as the dependant variable. The figure below shows only the significant trends — which happen to show that the legitimising frames become more salient, while the salience of all other frames stays the same.

Figure 6.11: Percent of Total Articles per Year Containing a Frame, Significant Trends Highlighted

So how do British newspapers frame protest? By using the six delegitimising and legitimising frames and one frame that fit neither of these two categories. And how did the framing change over time? The delegitimising framing stayed roughly on the same level. Yet, legitimising framing grew in importance over time.

Framing: Why?

The second analysis step was to ask why certain frames are employed over others in different circumstances. To do that, I combined my dataset of newspaper reports with the UK data on protest events from the Mass Mobilization Project (Clark and Regan 2019). I used multilevel logistic regression with random effects to determine the influence of different variables. Below I present the Average Marginal Effects for the significant relationships between my variables and the information if one of the frames from my two categories is present in an article.

Figure 7.2: Average Marginal Effects for Significant Variables in Aggregated Frame Models

This first important insight this analysis provides is that almost none of the variables have a significant influence on the usage of delegitimising frames. This basically confirms the notion of the protest paradigm being the default theme in reporting about protest. The probability that delegitimising framing is used only changed when violence occurs (increase of 10%) and when a report was published long after a protest event (decrease of 13%).

The second insight is that the effects on legitimising and delegitimising framing are not diametrically opposed. Just because legitimising framing is more likely to be used for social issue protests does not mean that delegitimising framing is less likely to be used in these cases. The only factor where this is not true is violence. Here the probability for delegitimising goes up almost at the same level as the probability for delegitimising decreases.

Finally, legitimising framing is influenced by many more factors than delegitimising framing. Here we also see the influence of the often described sharp divide between tabloid and broadsheet news. The division between left and right newspapers, however, plays no significant role.

I also ran models for the individual frames, however, the results are hard to summarise in brevity. What was interesting to see here is that individual delegitimising frames are influenced by several factors, yet these effects cancel each other out. For example, while there is more Law & Order and Troublemakers framing during protests against police violence, the Decay of Morals and Nuisance frames are used less often. This means the number of delegitimising frames stays on about the same level overall.

So, which factors explain the choice of frames by the news media when covering domestic protest events? Prepare yourself for a somewhat unsatisfying answer: it depends.

So What?

What are the contributions of my thesis, on which I worked literally thousands of hours? After some final hard thinking, I believe that I extended our knowledge in several ways, besides certainly extending my own knowledge.

  • Broad dataset: by basing my analysis on a broad dataset, I think that I provide a more systematic scrutiny of protest coverage.
  • Theoretical framework: I extended the theoretical underpinnings of the protest paradigm by suggesting guidelines on how to interpret results. Based on the analysis I can say that despite a stable majority of delegitimising frames, strong framing effects appear rather unlikely — although this might be different for some individual events.
  • Method: I developed and tested a method to systematically assess framing in large corpora.
  • Empirical findings I: the first part of the analysis provided that a majority of articles on protest used delegitimising frames; however, there is a substantial and growing number of articles that employ legitimising frames (as well).
  • Empirical findings II: the second analysis step provided four main determinants for frame usage:
    • Violence: violent protests get more delegitimising coverage, and less legitimising coverage.
    • Goal of a protest: the goal of a protest matters for the kind of reporting it receives. Alas the relationships between frames and goals are complex and goals overall matter more for legitimising frames.
    • Tabloid vs. Broadsheet: protests receive less legitimising coverage from tabloid newspapers than from broadsheet outlets. Surprisingly, this is one of few effects outlets have on how protest is covered.
    • Time: reports published more recently have a higher chance of containing legitimising framing, while reports published longer after the start of an event have a lower chance of containing delegitimising framing

I hope you find this somewhat interesting and consider giving my thesis a read. My aim is to split this into articles for targeted at different audiences. So maybe look out for these if you are interested. Cheers.


Amenta, Edwin, Thomas Alan Elliott, Nicole Shortt, Amber Celina Tierney, Didem Türkoğlu, and Burrel Vann. 2017. “From Bias to Coverage: What Explains How News Organizations Treat Social Movements.” Sociology Compass 11 (3): e12460.
Chan, Joseph Man, and Chi-Chuan Lee. 1984. “Journalistic Paradigms of Civil Protests: A Case Study of Hong Kong.” In The News Media in National and International Conflict, edited by Andrew Arno, 183–202. A Westview Special Study. Boulder, Colo.: Westview Press.
Chong, Dennis, and James N. Druckman. 2007a. “A Theory of Framing and Opinion Formation in Competitive Elite Environments.” Journal of Communication 57 (1): 99–118.
———. 2007b. “Framing Theory.” Annual Review of Political Science 10 (1): 103–26.
Clark, David, and Patrick Regan. 2019. “Mass Mobilization Protest Data.” Harvard Dataverse.
Edgerly, L., A. Toft, and M. L. Veden. 2011. “Social Movements, Political Goals, and the May 1 Marches: Communicating Protest in Polysemous Media Environments.” The International Journal of Press/Politics 16 (3): 314–34.
Kilgo, Danielle K., and Summer Harlow. 2019. “Protests, Media Coverage, and a Hierarchy of Social Struggle.” The International Journal of Press/Politics 24 (4): 508–30.
Lee, Francis L. F. 2014. “Triggering the Protest Paradigm: Examining Factors Affecting News Coverage of Protests.” International Journal of Communication 8 (1): 2725–46.
Matthes, Jörg, and Matthias Kohring. 2008. “The Content Analysis of Media Frames: Toward Improving Reliability and Validity.” Journal of Communication 58 (2): 258–79.
McLeod, Douglas M. 2007. “News Coverage and Social Protest: How the Media’s Protect Paradigm Exacerbates Social Conflict.” J. Disp. Resol., 185–94.
McLeod, Douglas M., and J. K. Hertog. 1999. “Social Control, Social Change and the Mass Media’s Role in the Regulation of Protest Groups.” In Mass Media, Social Control, and Social Change, edited by David Demers and K. Viswanath, 305–30. Ames: Iowa State Univ. Pr.
Wasow, Omar. 2020. “Agenda Seeding: How 1960s Black Protests Moved Elites, Public Opinion and Voting.” American Political Science Review 114 (3): 638–59.
Wouters, Ruud. 2015. “Reporting Demonstrations: On Episodic and Thematic Coverage of Protest Events in Belgian Television News.” Political Communication 32 (3): 475–96.

  1. Notation: P: removed punctuation; N: removed numbers; L: lowercased all words; S: stemming all words; W: removal of stopwords; I: removed infrequently used terms; 3: included bi- and trigrams; T: weighted words using term frequency by inverse document frequency (tf-idf)↩︎