How Do You Evaluate Qualitative Research Without Applying Quantitative Standards to It?

What Does “Good” Qualitative Research Actually Look Like?

You are critically reading an interview study. There is no power calculation, no control group, no p-values, and perhaps only 20 participants. Does that make the study weak?

Not necessarily. Those criticisms would matter if the study were making claims that required those features. But qualitative research often asks different questions and produces a different kind of evidence. It may investigate how people experience a phenomenon, how they interpret events, how practices unfold in particular contexts, or how meanings and social processes are constructed.

The opposite mistake is equally problematic. Saying that qualitative research is different does not mean that anything goes. Qualitative studies can have poorly justified samples, superficial data, incoherent analyses, unsupported interpretations, inadequate reflexivity, or conclusions that extend well beyond the evidence.

Critical appraisal therefore requires standards appropriate to the methodology. The question is not whether a qualitative study behaves like quantitative research. It is whether the researchers made coherent, transparent, and defensible methodological choices for the kind of knowledge they were trying to produce.

Evaluate Qualitative Research on Its Own Methodological Logic

What Should You Actually Look for in a Qualitative Study?

Start With the Question the Study Is Trying to Answer

Before judging the methods, determine what kind of question the researchers are asking. Qualitative methods are particularly useful when researchers seek detailed understanding of experiences, meanings, perspectives, interactions, practices, processes, or contexts that cannot be adequately represented by numerical measurement alone.

Suppose researchers want to understand how first-generation university students experience belonging during their first semester. Interviews, focus groups, observations, diaries, or other qualitative approaches may provide evidence that a standardized belonging score cannot capture on its own.

That does not make qualitative evidence inherently better. It makes it suited to a different inferential purpose. Your first appraisal question should therefore be whether the study design actually addresses the research question.

Do Not Treat “Qualitative” as a Single Method

Qualitative research is an umbrella term covering diverse methodological traditions and analytical approaches. Ethnography, grounded theory, phenomenological approaches, qualitative case studies, narrative inquiry, qualitative content analysis, and different forms of thematic analysis may have substantially different purposes and assumptions.

This matters because a criterion that is sensible for one approach may be inappropriate for another. For example, prolonged engagement in a field setting may be central to ethnographic work but irrelevant to a study based on existing documents. Theory generation may be an explicit purpose of some grounded theory approaches but not of a descriptive qualitative study.

Begin by identifying what methodology or analytical approach the authors claim to use. Then ask whether their procedures are coherent with that approach rather than comparing the study with an imagined generic version of “qualitative research.”

Methodological Coherence Matters More Than Resemblance to Quantitative Research

A useful way to read a qualitative study is to trace the logic connecting its major decisions:

What did the researchers want to understand? Why was the chosen methodology appropriate? Why were these participants, cases, documents, or settings selected? How were data generated? How were they analyzed? How did the researchers move from raw material to interpretations? What claims are ultimately being made?

These decisions should make sense together. A mismatch somewhere in that chain deserves attention even if every individual method sounds respectable in isolation.

Qualitative Sampling Is Usually About Relevance, Not Statistical Representativeness

Many qualitative studies deliberately select participants because they have particular experiences, characteristics, roles, or knowledge relevant to the phenomenon. Purposive sampling, theoretical sampling, criterion-based sampling, maximum-variation strategies, and other approaches may therefore be entirely appropriate.

A sample does not become weak merely because it is not statistically representative of a population. If the purpose is to understand how emergency nurses experience a particular clinical process, deliberately recruiting nurses with relevant experience may be more informative than drawing a random sample of the general population.

What you should examine is whether the sampling strategy fits the research aim. Who was included? Who was absent? Why were these cases selected? Did the sampling provide the variation or specificity required by the study? Could recruitment have systematically restricted the perspectives available?

These questions complement the broader task of evaluating whether the sample was appropriate, but the criteria must reflect the purpose of qualitative sampling.

A Small Qualitative Sample Is Not Automatically a Weak Sample

A sample of 15 or 20 interview participants may look alarmingly small if viewed through the assumptions of a population survey. That comparison is often inappropriate.

Qualitative sample adequacy depends on factors such as the study aim, specificity of the sample, richness and relevance of the material, methodological approach, quality of dialogue, and analytical strategy. Malterud and colleagues proposed the concept of information power: the more information relevant to the study aim that the sample contains, the fewer participants may be required.

Sample size still needs justification. The point is not that numbers never matter, but that a universal quantitative threshold does not determine qualitative adequacy. A small sample may provide rich evidence for a focused question, while a numerically larger sample can still provide shallow or poorly targeted data.

Be Careful With “Saturation”

You will frequently encounter statements such as “interviews continued until saturation was reached.” Do not treat that sentence as self-validating.

Saturation has been conceptualized and operationalized in different ways across qualitative methodologies. Researchers may mean that no new codes appeared, that additional data no longer changed developing categories, or something else entirely. The concept therefore needs methodological context rather than being used as a ritual phrase that automatically establishes sample adequacy.

If authors invoke saturation, ask what they mean by it, how they assessed it, what exactly was considered saturated, and whether that interpretation fits their analytical approach. In some qualitative traditions, saturation may not be the most appropriate principle for determining sample adequacy at all.

Examine How the Data Were Actually Generated

Interviewing people is not automatically rigorous qualitative research. Neither is conducting focus groups or collecting open-ended responses.

Look closely at the circumstances in which data were generated. Who conducted the interviews or observations? What was the relationship between researchers and participants? Where did data collection occur? Were interviews recorded and transcribed? Was the interview guide appropriate to the research question? Were questions excessively leading? Did the researchers have enough opportunity to obtain detailed accounts rather than brief surface-level responses?

For observational research, consider what was observed, for how long, in what settings, and how observations were recorded. For document or online-data research, examine how materials were selected and contextualized.

Reporting frameworks such as the Consolidated Criteria for Reporting Qualitative Research, or COREQ, explicitly include information about the research team and reflexivity, study design, data collection, analysis, and reporting because these details help readers understand how the evidence was produced.

Reflexivity Is Not an Admission That the Research Is Biased

In qualitative inquiry, researchers may be involved in producing and interpreting the data rather than functioning as invisible measuring instruments. Their disciplinary backgrounds, assumptions, identities, relationships with participants, theoretical commitments, and positions within the research setting may shape what questions are asked, what participants disclose, what researchers notice, and how material is interpreted.

Reflexivity involves examining those influences and making relevant aspects of the researcher's position and interpretive role visible.

Simply adding a reflexivity paragraph does not guarantee rigor. Ask whether the authors identify influences that genuinely matter and demonstrate awareness of their implications for the study.

Look for a Transparent Path From Data to Interpretation

One of the most important appraisal questions is deceptively simple: how did the researchers get from what participants said, did, or produced to the findings reported in the paper?

A statement such as “the transcripts were thematically analyzed” is usually insufficient by itself. You should be able to understand, at a level appropriate to the analytical approach, how researchers engaged with the material, developed codes or other analytical units where relevant, generated categories, themes, concepts, narratives, or interpretations, and refined their analysis.

Some approaches involve multiple coders or comparisons among analysts. Others do not treat coder agreement as a marker of quality because interpretation is conceptualized differently. The important question is whether the analytic procedures are coherent with the claimed methodology and sufficiently transparent for readers to understand how the findings were produced.

Do Not Automatically Demand Inter-Rater Reliability

Researchers trained primarily in quantitative methods sometimes expect qualitative coding to demonstrate inter-rater reliability or agreement coefficients. Such procedures can be useful in some forms of qualitative content analysis or projects where coding consistency is part of the methodological design.

They are not universal requirements for qualitative rigor. In interpretive approaches, researchers may explicitly reject the assumption that there is one objectively correct coding of a passage against which coders should converge.

Rather than asking whether every study reports a reliability coefficient, ask whether its approach to analysis, researcher interpretation, collaboration, disagreement, and quality assurance makes sense within the methodological framework being used.

Quotations Are Evidence, but Quotations Alone Are Not Analysis

Participant quotations can help readers see how an interpretation relates to the underlying material. COREQ, for example, includes the presentation of supporting quotations among its reporting criteria for interview and focus-group research.

But a paper does not become rigorous merely by filling the Results section with vivid quotations. The researchers still need to do analytical work.

Ask whether quotations or other data extracts actually support the interpretation, whether contradictory or more complex cases are acknowledged, whether the analysis moves beyond paraphrasing what participants said, and whether the researchers distinguish their interpretation from the raw material on which it is based.

Look for Complexity, Including Evidence That Does Not Fit Neatly

Strong qualitative analysis often takes variation seriously. Participants may disagree. Experiences may differ across contexts. A theme may apply strongly to some cases but poorly to others. A process may contain contradictions.

Findings that are suspiciously tidy deserve scrutiny, particularly when the phenomenon itself is complex. Ask whether the researchers considered alternative interpretations, divergent cases, tensions, or exceptions where these were relevant.

This does not mean every paper must contain a section labeled “negative cases.” The broader issue is whether the analysis appears genuinely responsive to the data or whether evidence has been arranged primarily to support a predetermined story.

Generalization Works Differently in Different Qualitative Studies

A qualitative study with 20 participants generally cannot support a claim such as “72% of university students experience this problem” unless an appropriate quantitative design supplies that estimate. But it does not follow that qualitative findings can say nothing beyond the participants who were studied.

Different qualitative traditions conceptualize the reach of findings differently. Researchers may discuss transferability to comparable contexts, theoretical or conceptual generalization, mechanisms or processes that may operate elsewhere, or detailed contextual knowledge that readers can assess for relevance to another setting.

The appropriate question is therefore not simply “Can this be generalized?” Ask what kind of inference the authors are making, to what settings or concepts, and whether the evidence and study design support that extension.

Reporting Guidelines Can Help, but They Are Not Quality Scores

The Standards for Reporting Qualitative Research, or SRQR, contains 21 reporting items intended to improve transparency across qualitative research. COREQ provides a 32-item checklist specifically developed for interviews and focus groups and organizes its criteria around the research team and reflexivity, study design, and data analysis and reporting.

These frameworks can be useful when appraising a paper because missing methodological information can make quality difficult to judge. But reporting and methodological quality are not identical. A well-reported methodological weakness remains a weakness, while an inadequately reported decision may be impossible to evaluate even if the underlying work was defensible.

Nor should every checklist be applied mechanically to every qualitative methodology. SRQR was deliberately developed to accommodate variation across qualitative approaches, while COREQ has a more specific scope. The appropriate reporting framework depends on the study being examined.

Why 18 Interviews Can Provide Stronger Evidence Than 100 Superficial Responses

Common Mistakes When Critically Appraising Qualitative Research

Use Methodological Fit Instead of a Quantitative Scorecard

When reading qualitative research, resist translating every familiar quantitative criterion into a qualitative equivalent. You do not need to find a qualitative version of statistical power, random sampling, inter-rater reliability, or population representativeness before you can decide whether the study is credible.

Instead, reconstruct the methodological argument of the paper. What did the researchers want to know? Why did they choose this approach? Who or what provided the data? How were those data generated? How did analysis produce the findings? What role did the researchers play? How well does the evidence support the interpretations?

Finally, distinguish a genuine methodological problem from a difference in methodological tradition. A feature should not be labeled a flaw merely because you would have made a different methodological choice. The more important question is whether the choice is defensible within the logic of the study and whether its consequences are acknowledged.

If you identify a weakness, then consider whether it is a limitation or a flaw serious enough to undermine the central inference. Qualitative studies, like quantitative studies, rarely divide neatly into “good” and “bad.”

What to Check When Evaluating Qualitative Research

Frequently Asked Questions About Evaluating Qualitative Research

Different Standards Do Not Mean Lower Standards

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.