After returning from fieldwork, Andre Stephens, Ph.D. candidate in Sociology, heard a question he’d asked of other researchers in the past, “How many interviews did you get?” Though, having been in the field for several months in both 2017 and 2018, Stephens now realized that the number of interviews completed was not as important as what one learned from them. In fact, it might not be useful to think of interviews as equally valuable, identically measured units of data, but as sites of observation for gathering data.
Stephens cautioned against using the standards of quantitative data analysis to assess interview data.
In this talk, entitled “Sizing Up the Sample: Interviews, Damned Interviews, and Statistics,” Stephens argued that interviewers should focus on saturation of information for the categories of social meaning in which they are interested. He supported this point with experiences during his time interviewing influential actors of Jamaica’s economic reforms, focusing on the IMF’s Extended Fund Facility agreement signed with the Jamaican government in 2013. First covering the different strengths of quantitative and qualitative data, he specifically challenged the assumptions that researchers should approach the latter with the standards of the former. Stephens encouraged researchers to contextualize information collected during interviews by investigating and considering the different amounts and kinds of capital held by the interviewee and their organization.
When judged by the standards of quantitative analysis, interview-based studies are ‘damned’ for their small sample sizes. At the start of his talk, Stephens encouraged interview researchers to consider that, while both kinds of data are useful for information and confirmation, qualitative data is especially good at informing us of what we do not know. Calls to make qualitative inquiry more rigorous often ignore the informative nature of the data. He emphasized that interviews are not individual representatives of the overall population, rather the population of interest is actually the universe of social knowledge and meaning. Thus, the interview is better conceived as a point of access to that universe.
In Stephens’s own research, the types of capital of his respondents (and their associated organizations) helped him to contextualize the information (and misinformation) he collected. For his study of the motives and dynamics of Jamaica’s economic reforms, he wanted to learn why the IMF would suddenly sign a new agreement with Jamaica in 2013, when, by the IMF’s own assessment, the country had not satisfactorily met the conditions of the previous agreement. The relevant forms of capital in this particular universe of social meaning were:
- technocratic/ideological
- regulatory/managerial
- financial/productive
- reputational/network power
Stephens’s respondents associated with the IMF relied on and reinforced their technocratic capital and their reputation as an independent institution in discussions of the agreement. Because of his understanding of the IMF’s reputation, he was skeptical about their articulated motivations for the new agreement. Through interviewees outside the IMF, Stephens learned that then prime minister of Jamaica had close ties to Maxine Waters, a prominent African-American Congresswoman and one of the most influential members of the Congressional Black Caucus under the Obama administration. He then learned (and it was confirmed by others) that the Caucus was ready to condemn the IMF for its austerity even in desperate circumstances. Waters’s influence with the Obama administration let them to insist that the IMF continue negotiations with Jamaica. Thus, the IMF went from refusing to meet with Jamaican officials to soon signing a new agreement. IMF-affiliated respondents did not mention the influence of the Congressional Black Caucus and the Obama administration however, as it would threaten the IMF’s reputation as an independent institution. Considering types of capital allowed an understanding of the IMF’s actions as an ideologically powerful actor, and that their respondents’ perspectives need to be understood at a presentation of their technocratic resources and reputation as independent of Washington D.C.
“…interviewers are best served to focus more on seeking to saturate their categories of interest rather than maximize the number of interviews they do.”
Stephens stressed that that point of this anecdote from his own fieldwork research was that interviews with actors with different forms of power and capital helped him to understand the process. In fact, he needed to speak with multiple actors to get the full story. Furthermore, he needed not speak with members of the Congressional Black Caucus themselves to understand their motivations and the situation. Stephens emphasized that this kind of information, which needed to be contextualized to be useful, was different from quantitative data, in which respondents are considered to be honest, expert representatives of themselves.
Stephens wrapped up his talk with visual representations of the analytical efficiency of qualitative and quantitative data based on the size of the sample and the amount of information collected. These figures visually demonstrated that “…interviewers are best served to focus more on seeking to saturate their categories of interest rather than maximize the number of interviews they do.”
To see Andre Stephens’s presentation from this QUAL Speaker Series talk, click here.
