Ethics and data transparency standards in the social sciences are changing and there is no telling where they will land in three or four years, says assistant professor of political science and QUAL Steering Committee member James Long. He set the context of his talk on Oct. 12 – the first of the QUAL Speaker Series for the 2016-17 school year – in experimental and quantitative research because “because those are the people driving the discussion.”
Long argued that the efforts of initiatives such as political science’s Data Access & Research Transparency (DART), combined with the rising mainstream interest in research ethics scandals, pose some unique challenges to qualitative researchers.
Scandals in the mainstream
Long set the stage with a series of examples, including an article in Vox, that have sparked an interest among mainstream audiences in what he dubs social science “scandals.” High profile cases have arisen from qualitative work as well, such as Alice Goffman’s book based on ethnographic study that came under fire last year – in both academic circles and in the media.
It is not simple to define these scandals, Long cautioned his audience made up primarily of Ph.D. students from international studies, political science, and anthropology. There are fundamental differences between cases of outright fraud and related, serious, but different issues such as interpretation, replication, and significance.
Why the ethics and data transparency discussion matters
Doctoral students from the Jackson School of International Studies, Political Science and other departments attended the ethics and data transparency talk.
The current ethics discussion in social science affects not only what researchers do and how they do it, but it also affects the people they study and their willingness to participate, noted Long.
“Researchers must communicate clearly and correctly the impact of research on the world,” he said. “[This is] especially tough with interpretation in qualitative research.”
Another implication for graduate students and scholars is publication of their research in peer-reviewed journals. Long summarized the Journal Editors’ Transparency Statement (JETS) from DART as having three main themes:
- data generation (how the evidence is gathered),
- data replication (how the conclusions or interpretations are drawn from the empirical data), and
- data access (the relationship between the sample data and the broad evidentiary record).
What this means for quantitative and experimental researchers is pretty straight forward and consistent with existing best practices for research design, Long argued. But what these three editors’ mandates mean for qualitative research is not as clear.
Long dismissed IRB (Institutional Review Board) as a tool for complying with DART – because they are concerned with different aspects of a research project. The former is concerned with ethical impact on the people studied, the latter is focused on the ethics and transparency of the research design, data collected, and analysis process.
Advice and resources
If IRB is not the answer to DART compliance for qualitative researchers, what should they do to address these guidelines and thus improve the chances of publication?
Part of the challenge, Long argued, is that different disciplines have different standards for conducting, reporting, and publishing research. With that caveat, Long discussed a few examples of qualitative research considerations that may be applicable for multiple disciplines.
Interviews
For some types of research where the data collection method is interviews, DART could be partially satisfied by keeping a table of interviews with categories such as the source’s title or role, length of interview, format, etc. One example of this simple approach comes from Jackson School Professor Robert Pekkanen’s 2015 article with Erik Bleich, “Data Access, Research Transparency, and Interviews: The Interview Methods Appendix.”
Assistant Professor of Political Science James Long discusses how interviews and textual data/archive research may be affected by emerging ethics guidelines for publication.
The DART compliance task becomes much more difficult when issues of confidentiality hinder what a researcher is able to report about their interviewees. This is a concern of IRB as well, but strict DART publishing requirements may have unintended consequences. If IRB protections are enforced for sensitive interviews, publication could still go forward by anonymizing data. However, if DART is strictly enforced as it currently stands, this type of research would not be easily publishable. Long fears this would eliminate some projects altogether: “So certain topics may not be researched or at least not researched in a qualitative way,” he said.
Qualitative researchers’ decisions about how – or even whether – to embark on certain types of projects are made even more difficult by the fact that DART guidelines for publications are not static, the discussion continues and the rules may change.
For more tips and a list of resources, see James Long’s slides from this talk:
