Methodology

“The West has over-relied on elections as instruments of accountability… Elections only work if we complement them with an informed society.”
– Oxford Economist Paul Collier

The Study’s Background

This study analyzes how 13 major news outlets in the United States, the United Kingdom and the Middle East covered elections in four different countries prominently in the news in the past five years: Afghanistan, Iran, Iraq and Pakistan.

The news outlets are as follows:

Al Jazeera (English) The New York Times
Christian Science Monitor NPR (National Public Radio)
The Economist Time magazine
The Financial Times The Times (London)
Fox News USA Today
The Guardian The Washington Post
NBC

Five research teams originally set out to consider whether there have been any significant differences in how major English-language news outlets—especially those with a global audience—report on what’s been called the “Muslim World.”

From an initial qualitative evaluation of about 25 news outlets across all platforms (print, television, radio, online) selected for their reach and their range, it quickly became apparent that every one of those news outlets tended to focus on the four countries sequentially.  In other words, when one country, say Iran, was in the news, coverage of the other three countries lagged  behind.

As a result, the research teams determined that understanding how the news media cover these countries would not be well-served by simply evaluating the coverage within a single range of time.  Instead, the research teams decided to select a compelling news event that had an analogue in all four countries and evaluate the coverage of that event when it occurred.  The event that was selected was the national election in each of the countries.

Following that decision, the research teams  whittled down the 25 news outlets to 13 that had significant international coverage for their media platform. Each team then individually determined the appropriate time period for gathering the coverage of that country’s election by considering not only when  the elections were held, but noting whether there were  significant related events before or after the elections.  If there were significant protests or violence, for example, leading up to or following an election that received international attention than that extended period of coverage was also evaluated.

As a result the research teams evaluated coverage over different periods of time, as follows:

  1. Iraq: February 13, 2010 to April 8, 2010
  2. Afghanistan: August 1, 2009 to October 31, 2009
  3. Iran: May 1, 2009 to August 31, 2009
  4. Pakistan: November 1 2007 to February 29, 2008
  5. Iraq: November 1, 2004 to January 31, 2005

Using Lexis-Nexis and Factiva, the research teams then moved to download all the coverage from the 13 news outlets on the expanded election time frame in each of the four countries.  They identified that coverage in each outlet by searching for the keywords “election” and ‘[country name].”

After extensive training on data and content analysis, the country research teams collated the downloaded transcripts and compiled spreadsheets of data arranged by news outlet and a variety of word choices, grouped in multiple categories as follows:

  1. Religion
  2. Protest & Politics
  3. Terror & Death terms
  4. Gender & Family
  5. Youth & Social Media terms

The selection of these categories emerged from an initial series of questions posed by the researchers:

  • Are elections in a Muslim country just another way for US and UK media to make judgments about Islam?
  • Do reporters assess  the role of elections in making a government more representative and more democratic?
  • Do media emphasize the implications of the election for stability and security in the region – or do they emphasize the implications for the home country of the media outlet ?
  • Do reporters cover an election in the Islamic world as a political horse race among candidates – as is often the case in elections in the United States or Britain?
  • Do reporters focus on the personalities of the candidates (the charisma of a Benazir Bhutto, for instance, or the “honesty” of a Mir-Hossein Mousavi) or on the policies the candidates support?
  • Are reporters concerned with individual voters?  To what extent are the demographics of the voters (young or female or educated, for example) explored?  Are journalists concerned with the civil rights of citizens?

Using IBM’s Collaborative User Experience research group platform ManyEyes, the teams created several types of data visualizations: from the spreadsheets they created bar graphs and stack graphs for categories, and from the collected transcripts they created word trees and phrase nets.  The evaluation of the coverage was expedited and made considerably more neutral and efficient through the ManyEyes tools.

The members of the research teams individually coded the news outlets.  Two research members then went through the original findings to check the evaluations and results.  Quantitative and qualitative results were reviewed and checked to confirm accuracy and consistency. In order to further ensure reliability, all the spreadsheets and analysis were coded and evaluated for a third time by the research team leader, Dr. Moeller.

Once the coding/spreadsheets/data analysis was finalized the results were visualized, formatted and made ready for review.

See the Data Visualization page for links to the spreadsheets and the ManyEyes visualizations.  Note that the data visualizations are dynamic and can be manipulated online.

The Research Team

Dr. Susan Moeller supervised the research at the Philip Merrill College of Journalism at the University of Maryland, College Park. Jennifer Memmolo and Erin Spiegel took lead roles in the research, analysis and writing/editing of the findings.  Other researchers were seniors journalism and communication students: Sunaina Arora, Kamilia Butler-Peres, Norman Carter, Hal Decoursey, Juliette Ebner, Krystle Idnani, Emily Mekinc, Caitlin O’Brien, Hannah Park, Kyle Patton, Katie Valavanis, Lauren White, Kate Yanchulis,

(See The Research Team for more details).

Acknowledgment:

Professor Susan Moeller would like to gratefully acknowledge the generous support and professional interest of the Carnegie Corporation of New York.  Its Carnegie Scholar award first inspired the research parameters of this study, and its funding allowed Prof. Moeller the time needed to turn draft data and analysis into publishable research.

Header Image Christopher Anderson / Magnum Photos
Advertisements