+ - 0:00:00
Notes for current slide
Notes for next slide

I started my PhD with this question in mind. The reason was this example.

Semi-automated Content Analysis of Media Frames

How to Analyse Media Reports (of protest)

Johannes B. Gruber

University of Glasgow

2019-06-22

1 / 19

How can we find systematic patterns in media reports

if the reported topics differ?

2 / 19

I started my PhD with this question in mind. The reason was this example.

How can we find systematic patterns in media reports

if the reported topics differ?

Application: Coverage of protest events over time

2 / 19

I started my PhD with this question in mind. The reason was this example.

How can we find systematic patterns in media reports

if the reported topics differ?

Application: Coverage of protest events over time

Idea: Analyse the framing of a story instead of its content

2 / 19

I started my PhD with this question in mind. The reason was this example.

I collected coverage containing reports about domestic protest from 26 UK newspaper. The topics of the protests differ wildly. Between fox-hunting protests, Anti-war protest, high fuel prices and pro- and anti-Brexit protests.

What is framing?

3 / 19

What is framing?

A frame is a "a central organizing idea or story line that provides meaning to an unfolding strip of events" (Gamson & Modigliani, 1987, p. 143)
3 / 19

We all use framing to make sense of everyday events and issues when telling others about it. Example: Campus West end: We can either tell people about the modern architecture blending in with the style of the old buildings; the bright rooms, modern teaching facilities, that the campus is adjacent to a beautiful park and that is nice food available on campus. Or we can tell people that it's almost impossible to live close to campus since it is located in one of the most expensive neighbourhoods in the city, that there are no cheap places nearby where you can eat or drink (tying students to the expensive and sometimes low quality food in the Mensa) and that there are not enough power outlets in the offices. None of these facts are untrue, but based on the selection of information, you can tell completely different stories

How can we detect and code frames?

  • Most often used definition of framing in media studies:

"To frame is to select some aspects of a perceived reality and make them more salient in a communicating text, in such a way as to promote a particular problem definition, causal interpretation, moral evaluation, and/or treatment recommendation for the item described" (Entman, 1993, original emphasis).

4 / 19

Available Approaches

  • Qualitative

  • Manual-Holistic

  • Manual-Clustering

  • Automated Content Analysis (ACA)
5 / 19

Available Approaches

  • Qualitative

  • Manual-Holistic

  • Manual-Clustering

  • Automated Content Analysis (ACA)

→ focus only on in-depth description
→ hard to ensure validity and reliability
→ easier to code = more validity and reliability
→ make analysis scalable but same concept?

5 / 19

Qualitative approaches

  • rooted in qualitative research traditions
  • proceed inductively
  • more about in-depth description
  • little or no quantification of elements or the distribution of frames within a discourse is provided by the researcher Manual-Holistic approaches
  • frames are holistic variables
  • usually quantitative content analyses
  • frames can be either derived from the literature or identified inductively in a pilot study of a small sample
  • validity and reliability depend on the transparency with which the study communicates the coding decisions manual-clustering approaches
  • split up frames into sub-variables which are easier to code in content analysis-
  • frames are operationalised as a set of yes/no indicator questions (coders are asked if a certain aspect is mentioned in the text or not)

Available Approaches

  • Qualitative

  • Manual-Holistic

  • Manual-Clustering

  • Automated Content Analysis (ACA)
    • Dictionary Methods (deductive)
    • Fully Automated Classification (inductive)
    • Supervised Machine Learning (SML)

→ focus only on in-depth description
→ hard to ensure validity and reliability
→ easier to code = more validity and reliability
→ make analysis scalable but same concept?

5 / 19

Qualitative approaches

  • rooted in qualitative research traditions
  • proceed inductively
  • more about in-depth description
  • little or no quantification of elements or the distribution of frames within a discourse is provided by the researcher Manual-Holistic approaches
  • frames are holistic variables
  • usually quantitative content analyses
  • frames can be either derived from the literature or identified inductively in a pilot study of a small sample
  • validity and reliability depend on the transparency with which the study communicates the coding decisions manual-clustering approaches
  • split up frames into sub-variables which are easier to code in content analysis-
  • frames are operationalised as a set of yes/no indicator questions (coders are asked if a certain aspect is mentioned in the text or not)

How can we detect and code frames (better)?

  • Most often used definition of framing in media studies:

"To frame is to select some aspects of a perceived reality and make them more salient in a communicating text, in such a way as to promote a particular problem definition, causal interpretation, moral evaluation, and/or treatment recommendation for the item described" (Entman, 1993, original emphasis).

6 / 19

How can we detect and code frames (better)?

  • Most often used definition of framing in media studies:

"To frame is to select some aspects of a perceived reality and make them more salient in a communicating text, in such a way as to promote a particular problem definition, causal interpretation, moral evaluation, and/or treatment recommendation for the item described" (Entman, 1993, original emphasis).

  • Instead of coding frames, I code frame elements
6 / 19

Method (1): Finding Frames

7 / 19

Method (2): Replicating frames

8 / 19

Method Alternative

9 / 19

Application: Background

  • Case/population: Mainstream news media articles about protests in the UK (1992-2017)

  • Time-series design: it is expected that the patterns have changed substantially since the first seminal studies – not least due to the arrival of the internet (Cottle, 2008)

  • Data: Population scale sample of protest reports in newspapers (n > 27,000)

  • State of knowledge: Journalists use a default theme (so called protest paradigm) to report about protest: details about the event (clash with police, the appearance of protesters, nuisance caused or reactions of bystanders) are highlighted while the message of protesters is undermined or not even mentioned.

10 / 19

Application: Codebook

Frame elements are further divided into coding variables:

  • Problem Definition
    • Topic
    • Actor
  • Causal Attribution
    • Benefit Attribution
    • Risk Attribution
  • Moral Evaluation
    • Benefit
    • Risk
  • Treatment
    • Judgement
11 / 19

Application: Codebook

12 / 19

Application: Coding example

«Police worked very hard with the organisers to ensure a peaceful protest and it was a small core of determined troublemakers bent on conflict with the police who I believe were responsible for the violence. There were fireworks and missiles thrown at the police and some people were intent on breaking through the barriers. We were there to ensure that did not take place.»

  • Topic: Violence/Crime
13 / 19

Application: Coding example

«Police worked very hard with the organisers to ensure a peaceful protest and it was a small core of determined troublemakers bent on conflict with the police who I believe were responsible for the violence. There were fireworks and missiles thrown at the police and some people were intent on breaking through the barriers. We were there to ensure that did not take place.»

  • Topic: Violence/Crime
  • Actor: Police
13 / 19

Application: Coding example

«Police worked very hard with the organisers to ensure a peaceful protest and it was a small core of determined troublemakers bent on conflict with the police who I believe were responsible for the violence. There were fireworks and missiles thrown at the police and some people were intent on breaking through the barriers. We were there to ensure that did not take place.»

  • Topic: Violence/Crime
  • Actor: Police
  • Benefit: Reinstating public order
13 / 19

Application: Coding example

«Police worked very hard with the organisers to ensure a peaceful protest and it was a small core of determined troublemakers bent on conflict with the police who I believe were responsible for the violence. There were fireworks and missiles thrown at the police and some people were intent on breaking through the barriers. We were there to ensure that did not take place.»

  • Topic: Violence/Crime
  • Actor: Police
  • Benefit: Reinstating public order
  • Risk: Public Safety
13 / 19

Application: Coding example

«Police worked very hard with the organisers to ensure a peaceful protest and it was a small core of determined troublemakers bent on conflict with the police who I believe were responsible for the violence. There were fireworks and missiles thrown at the police and some people were intent on breaking through the barriers. We were there to ensure that did not take place.»

  • Topic: Violence/Crime
  • Actor: Police
  • Benefit: Reinstating public order
  • Risk: Public Safety
  • Benefit Attribution: Police
13 / 19

Application: Coding example

«Police worked very hard with the organisers to ensure a peaceful protest and it was a small core of determined troublemakers bent on conflict with the police who I believe were responsible for the violence. There were fireworks and missiles thrown at the police and some people were intent on breaking through the barriers. We were there to ensure that did not take place.»

  • Topic: Violence/Crime
  • Actor: Police
  • Benefit: Reinstating public order
  • Risk: Public Safety
  • Benefit Attribution: Police
  • Risk Attribution: Protesters
13 / 19

Application: Coding example

«Police worked very hard with the organisers to ensure a peaceful protest and it was a small core of determined troublemakers bent on conflict with the police who I believe were responsible for the violence. There were fireworks and missiles thrown at the police and some people were intent on breaking through the barriers. We were there to ensure that did not take place.»

  • Topic: Violence/Crime
  • Actor: Police
  • Benefit: Reinstating public order
  • Risk: Public Safety
  • Benefit Attribution: Police
  • Risk Attribution: Protesters
  • Judgement: None
13 / 19

Application: Coding example (2)

Par_ID Problem Definition: Topic: Violence/Crime Problem Definition: Actor: Police Moral Evaluation: Benefit: Reinstating public order Moral Evaluation: Risk: Public safety Causal Attribution: Risk_Attribution: Protesters Causal Attribution: Benefit_Attribution: Police Treatment: Judgement_Positive: 0 Treatment: Judgement_Positive: 1 Problem Definition: Topic: Nuisance Problem Definition: Topic: Protesters …
14900405 1 1 1 1 1 1 0 0 0 0 0
14 / 19

Application: Clustering Frame Elements

The R package NbClust (Charrad et al., 2014) combines many indices to determine optimal cluster solutions:.

15 / 19

"ch" (Calinski and Harabasz 1974) "duda" (Duda and Hart 1973) "pseudot2" (Duda and Hart 1973) "cindex" (Hubert and Levin 1976) "gamma" (Baker and Hubert 1975) "beale" (Beale 1969) "ccc" (Sarle 1983) "ptbiserial" (Milligan 1980, 1981) "gplus" (Rohlf 1974; Milligan 1981) "db" (Davies and Bouldin 1979)

Application: Interpreting Clusters as Frames

Heatmap showing cluster means for codes:

16 / 19

Application: SML replicating classification

model Accuracy AccuracyLower AccuracyUpper package
Maximum Entropy 0.64 0.44 0.81 RTextTools
SVM 0.59 0.39 0.78 quanteda.classifiers
LogitBoost 0.59 0.36 0.79 caret/caTools
bagging 0.50 0.31 0.69 RTextTools
Naive Bayes 0.48 0.29 0.68 quanteda
Random Forest 0.48 0.29 0.68 caret/ranger
NNSEQ 0.44 0.25 0.65 quanteda.classifiers
Penalised Multinomial Regression 0.44 0.25 0.65 glmnet

Work in progress (training/test sample n = 270/30)!

17 / 19

Once this is done, we can show change over time and do some analysis why certain reports are the way they are (right wing protest, more positive reports from right-wing media?)

Application: Next Steps

  • Finish training sample
  • Agreement between coders on training set
  • Agreement between coders and clustering
  • Agreement between clustering and SML
  • Outlook:
    • Explain the framing of protest with event data (size, tactics, time after event, ideogical stance, etc.)
18 / 19

Conclusion

  • Detected frames make sense
19 / 19

Conclusion

  • Detected frames make sense
  • Classification better than chance already
19 / 19

Conclusion

  • Detected frames make sense
  • Classification better than chance already
  • More control over categories than topicmodels
  • Less abstract coding and category building than dictionary methods
19 / 19

Thank you for your attention!

 

Working Paper

bit.ly/JBGruber_framing_paper

Contact

Johannes B. Gruber

  • Mail: j.gruber.1@research.gla.ac.uk
  • Web: johannesbgruber.eu/
  • GitHub: github.com/JBGruber
  • Twitter: @JohannesBGruber
19 / 19

Method

19 / 19

Dataset construction

Data downloaded from LexisNexis using "protest" and "demonstration" (plus several variations) before cleaning the data:

19 / 19

Frames in newspapers

19 / 19

How can we find systematic patterns in media reports

if the reported topics differ?

2 / 19

I started my PhD with this question in mind. The reason was this example.

Paused

Help

Keyboard shortcuts

↑, ←, Pg Up, k Go to previous slide
↓, →, Pg Dn, Space, j Go to next slide
Home Go to first slide
End Go to last slide
Number + Return Go to specific slide
b / m / f Toggle blackout / mirrored / fullscreen mode
c Clone slideshow
p Toggle presenter mode
t Restart the presentation timer
?, h Toggle this help
Esc Back to slideshow