• Authority. No. Not all authorities are created equal. Some authorities are right, other authorities are wrong. Therefore, authority is not a source of evidence for a scientific research study unless they support their statements with valid observations based on scientific research. This is important to remember when developing a questionnaire. In most cases, authorities cannot report on the behavior of another person or the effectiveness of a program.
• Tradition. No. Some traditions are good and effective, other traditions are either bad (e.g., killing twins) or ineffective (e.g., rain dances). Traditions can be the object of a scientific study. For example, a researcher might conduct a scientific study to determine whether massaging babies really does help them to grow faster. But traditions cannot be used as a source of evidence for scientific research.
• Common Sense. No. Like traditions, some common sense behaviors are effective and others are ineffective. Common sense beliefs can also be the object of a scientific study. Researchers need to conduct research in Nigeria to determine whether beating students really is effective in the long-term.
• Observation. Yes. Observation is the only valid source of evidence for scientific research. The purpose of the Methods section of a research study is to identify research instruments and procedures that will be used to collect observations that are valid and useful for answering a research question or hypothesis.

There are three key principles that determine whether a research question qualifies as scientific research.

1. Empirical. Scientific questions are answered through systematic observation. This means that the question must be answered by data that is experienced directly or indirectly through the five senses. Direct observation is something that the scientist can clearly see, such as height. However, not everything can be seen directly, such as a person's intelligence. This requires indirect observation, such as a score on an intelligence test.
2. Verifiable. Observations must be collected under specific conditions that can be confirmed by all normal individuals. Replication of scientific observations is a foundational principle in science: data is not valid unless it can be duplicated by other researchers. This means that researchers need to clearly articulate the exact research design, sample and sampling procedures, instruments, data collection procedures, and data analyses that were used so any other researcher can exactly replicate the results to verify their authenticity.
3. Control. Scientific observations must be collected using methods that attempt to control the conditions of the study in the following ways:
   • Eliminate sources of bias. Research can be biased in the sample chosen so it is not representative of the population, biased in the way that items are worded on the instrument, or biased in the procedures for data collection. More on each of these sources of bias will be discussed in their appropriate chapters.
   • Reduce error that may distort the results. Error can be introduced into a study because the questionnaire is poorly designed, the questionnaire was administered in undesirable conditions, or the questionnaire was poorly scored and analyzed.
   • Rule out all other alternate explanations of the findings. There can be multiple ways to explain research results. For example, a researcher might conclude that a computerized instruction was effective because students in the treatment group performed better on a test than students in the control group. However, an alternate explanation might be that different teachers were used in the two groups, and students in the treatment had a better teacher. To control for this alternative explanation, the researcher must thoughtfully plan the study's methods to control for all other explanations, perhaps by using the same teacher for both groups.

Conducting scientific research requires considerable thought and planning in advance of collecting observations (subsequently called data). Thus, it must be emphatically noted that a researcher, particularly a beginning researcher, MUST plan the study's methods well in advance of collecting data. Planning a study will take weeks and months of planning the methods and developing the instrument. However, the outcome of a well-planned and well-conducted research study is both informative and useful for developing excellent educational practices. Educational research is an exciting endeavor that has the potential to greatly improve society.

NEXT

Return to Educational Research Steps

Copyright 2012, Katrina A. Korb, All Rights Reserved