Should You Prefer an Existing Measure or Create a New One?

Should You Use a Measure That Already Exists or Build Your Own?

You have defined your construct and need a way to measure it. A literature search reveals several existing questionnaires, but none seems perfect. Perhaps the wording feels dated, the population differs from yours, the instrument is too long, or your construct has a feature the available measures do not appear to capture.

Creating your own instrument can seem attractive. You can write exactly the items you want and tailor them to your study. Yet developing a defensible measure involves considerably more than writing a set of plausible questions and calculating Cronbach's alpha afterward.

In many situations, an appropriate existing measure is the stronger starting point. A new measure is most defensible when there is a genuine measurement need that existing instruments cannot adequately meet and you have the resources to develop and evaluate the new measure properly.

Prefer an Appropriate Existing Measure, but Do Not Use One Merely Because It Exists

Choose the Measurement Strategy Before You Start Writing Items

Search for Existing Measures Before Developing a New One

Before creating an instrument, conduct a systematic enough search to determine what already exists. Relevant measures may appear in empirical studies, systematic reviews of measurement instruments, instrument databases, methodological papers, supplementary materials, manuals, or original development and validation articles.

Searching only for the exact wording of your construct can miss useful candidates because different research traditions may use related terminology. Examine the construct literature first, then search synonyms, dimensions, neighboring terms, and established theoretical frameworks.

The goal is not merely to find a questionnaire. It is to determine whether a suitable measurement approach already exists and what evidence supports it.

An Existing Measure Offers More Than Convenience

A well-established measure may come with a conceptual framework, documented item-development process, scoring rules, reliability evidence, validity evidence, normative or comparative information, translations, and experience across multiple studies.

Using the same instrument as previous research can also facilitate comparison or synthesis when populations, administration, scoring, and interpretations are sufficiently comparable.

These advantages can save substantial work. More importantly, they allow your measurement decisions to build on an existing body of evidence rather than beginning from zero.

However, “existing” does not mean “appropriate,” and “widely used” does not mean “best.”

Do Not Choose an Instrument by Popularity Alone

A measure can accumulate hundreds of citations because it was available early, easy to administer, familiar to reviewers, or repeatedly inherited from previous studies. None of those facts independently establishes that it is the most appropriate instrument for your research question.

Read the original development paper or manual where available. Determine how the construct was defined, how items were generated, which dimensions are represented, how scores are calculated, what population was involved, and what evidence supports the intended interpretation.

Then compare that information with your own conceptual and operational definitions.

Evaluate the Construct Before the Psychometric Coefficients

Researchers sometimes compare candidate instruments by looking first for Cronbach's alpha, factor loadings, or another familiar statistic. Those properties matter, but they cannot rescue an instrument that measures the wrong construct.

Begin with content. Does the measure represent the construct as you define it? Are important dimensions missing? Does it include content that belongs to something else?

Content validity is therefore an important consideration in instrument selection. COSMIN, for example, places particular emphasis on whether items are relevant, comprehensive, and comprehensible for the construct, population, and context of use.

If an existing instrument captures only part of the construct you need, a strong reliability coefficient does not solve the mismatch.

Population and Context Matter

An instrument developed with working adults in one country may not automatically function identically among adolescents in another language and educational system. A measure designed for clinical screening may not support the same interpretation when used for population research. A technology-use scale developed for one generation of tools may contain content that becomes outdated.

This does not mean that every change of setting requires developing a new instrument. It means you should examine whether the available evidence is relevant to your intended use and whether additional evaluation is needed.

The more specific question of how to determine whether a measure is appropriate for your population and context should therefore be part of instrument selection.

Adaptation Is Not the Same as Using the Existing Measure Unchanged

Researchers often say they “used” an established scale after changing its wording, deleting items, altering response categories, translating it, replacing the referent, changing the recall period, or administering it through a different mode.

Some modifications may be entirely reasonable. They can also change item meaning, construct coverage, response processes, score comparability, or measurement properties.

Translation Requires More Than Replacing Words

When an instrument is used in another language, literal translation alone may not preserve meaning. Terms, response categories, idioms, examples, and assumptions can function differently across languages and cultures.

International Test Commission guidelines emphasize systematic procedures for adapting tests across linguistic and cultural contexts rather than treating translation as a purely lexical task. Depending on the instrument and intended use, adaptation may involve multiple translators, expert review, target-population input, cognitive interviewing or pretesting, and empirical evaluation.

The appropriate procedure depends on the stakes and measurement context, but the basic principle is stable: translated wording should support the intended construct interpretation, not merely resemble the original sentence.

When Is a New Measure Justified?

Developing a new measure may be warranted when the construct itself is genuinely new, existing measures operationalize it in ways inconsistent with your theoretical definition, important dimensions are absent, available instruments are unsuitable for the intended population or context, or existing measures cannot support the specific intended use.

A new measure may also be justified when existing instruments are inaccessible or impractical, although practical inconvenience alone should be weighed against the substantial work required to establish a defensible alternative.

The key is to identify the measurement gap. “I wanted my own questionnaire” is not a measurement gap.

Developing a Measure Is a Research Project, Not a Formatting Task

Writing items is only one part of instrument development. Depending on the construct and purpose, development may involve defining the construct and dimensions, reviewing existing measures, generating an item pool, obtaining substantive and target-population input, evaluating content, studying response processes, piloting items, examining dimensionality, estimating reliability, evaluating relationships with other variables, assessing measurement invariance or other properties where relevant, and refining scoring and interpretation.

The exact process depends on the measurement model and intended use. There is no single universal sequence that every scale must follow.

What should be avoided is the familiar shortcut in which researchers write a handful of questions, administer them to the study sample, obtain an acceptable alpha, and declare the questionnaire “validated.” Internal consistency alone cannot establish that the items adequately represent the construct or that the resulting scores support the intended interpretation.

Your Main Study Sample Should Not Be an Afterthought in Instrument Development

If developing a new measure is itself a substantial methodological objective, plan the validation work explicitly. Exploratory and confirmatory analyses, cross-validation, item refinement, and evaluation across relevant groups may require sample sizes and designs beyond those originally planned for the substantive research question.

Repeatedly modifying a measure until it behaves well in one dataset can capitalize on sample-specific characteristics. Independent evidence or appropriately designed validation procedures can therefore be important before treating the final measurement model as established.

Shortening an Existing Scale Creates a New Measurement Question

Researchers sometimes remove items simply because a questionnaire is too long. Reducing participant burden is a legitimate concern, but shortening a scale can alter reliability, content coverage, dimensionality, and score interpretation.

When brevity is necessary, first look for an established short form or evidence-supported alternative. The trade-off between single-item and multi-item measurement should be addressed explicitly rather than solved by deleting whichever items seem repetitive.

Permissions, Licensing, and Conditions of Use Matter

Not every published instrument is automatically free to reproduce, translate, modify, or distribute. Some instruments are openly available; others require permission, licensing, fees, registration, or specific conditions of administration.

Verify the current terms with the instrument owner, publisher, official website, or manual rather than assuming that publication in a journal places the instrument in the public domain.

Do this early. Discovering after data collection that an instrument was used outside its permitted conditions is an administrative headache of the sort methods sections rarely warn you about.

When an Existing Scale Is Close but Not Quite Right

The decision is therefore not “existing instruments are rigorous; new instruments are innovative.” Either can be appropriate. The stronger option is the one supported by the clearest measurement argument and adequate evidence.

Common Mistakes When Choosing or Developing a Measure

Choose Among Use, Adaptation, and New Development Deliberately

Once you have searched the literature and identified plausible instruments, compare them against the measurement requirements of your study before deciding to create something new.

Do not force the decision before examining the candidates. Sometimes the literature search reveals that an existing instrument is a surprisingly good fit. Sometimes it reveals that the construct has been operationalized in several incompatible ways. That is useful information in its own right because it tells you where the real measurement problem lies.

Before Creating a New Measure, Check Whether You Actually Need One

Questions About Existing and Newly Developed Measures

Do Not Develop a New Measure Until You Know What Existing Measures Cannot Do

How to Cite This Guide

This guide is intended to be read, shared, and used in research, teaching, and academic work. If you draw on its ideas, explanations, or other content, please acknowledge the source by citing the guide. Doing so gives appropriate credit and helps your readers locate the original resource.