Validity, reliability, accuracy and precision

• Validity: variables controlled so that any measured effect is likely to be due to the independent variable.
• Reliability: consistent values in repeats and independent replicates.
• Accuracy: data, or means of data sets, are close to the true value.
• Precision: measured values are close to each other.

(a) Pilot study

• Integral to the development of an investigation, a pilot study is used to help plan procedures, assess validity and check techniques
• This allows evaluation and modification of experimental design
• The use of a pilot study can ensure an appropriate range of values for the independent variable
• In addition, it allows the investigator to establish the number of repeat measurements required to give a representative value for each independent datum point

(b) Experimental design

(i) Independent and dependent variables

• An independent variable is the variable that is changed in a scientific experiment.
• A dependent variable is the variable being measured in a scientific experiment.
• Independent and dependent variables can be continuous or discrete
• Experiments involve the manipulation of the independent variable by the investigator
• The experimental treatment group is compared to a control group
• The use and limitations of simple (one independent variable) and multifactorial (more than one independent variable) experimental designs
• The control of laboratory conditions allows simple experiments to be conducted more easily than in the field.
• However, a drawback of a simple experiment is that its findings may not be applicable to a wider setting.
• A multifactorial experiment involves a combination of more than one independent variable or combination of treatments.
• Investigators may use groups that already exist, so there is no truly independent variable
• Observational studies are good at detecting correlation, but since they do not directly test a hypothesis, they are less useful for determining causation
• In observational studies the independent variable is not directly controlled by the investigator, for ethical or logistical reasons.

(ii) Confounding variables

• Due to the complexities of biological systems, other variables besides the independent variable may affect the dependent variable
• These confounding variables must be held constant if possible, or at least monitored so that their effect on the results can be accounted for in the analysis
• In cases where confounding variables cannot easily be controlled, a randomised block design could be used
• Randomised blocks of treatment and control groups can be distributed in such a way that the influence of any confounding variable is likely to be the same across the treatment and control groups.

 (iii) Controls

• Control results are used for comparison with the results of treatment groups
• Negative and positive controls may be used
• The negative control provides results in the absence of a treatment.
• A positive control is a treatment that is included to check that the system can detect a positive result when it occurs.

Use of placebos and the placebo effect

• Placebos can be included as a treatment without the presence of the independent variable being investigated.
• Placebo effect is a measurable change in the dependent variable as a result of a patient’s expectations, rather than changes in the independent variable.

(iv) In vivo and in vitro studies

• In vitro refers to the technique of performing a given procedure in a controlled environment outside of a living organism
• Examples of in vitro experiments: cells growing in culture medium, proteins in solution, purified organelles.
• In vivo refers to experimentation using a whole, living organism

Advantages and disadvantages of in vivo and in vitro studies

• In vitro: easier to control confounding variables but uncertain if results are applicable to whole organisms.
• In vivo: more difficult to control confounding variables, but any results are more applicable to whole organisms/in a wider setting.

 (c) Sampling

• Where it is impractical to measure every individual, a representative sample of the population is selected
• The extent of the natural variation within a population determines the appropriate sample size
• More variable populations require a larger sample size
• A representative sample should share the same mean and the same degree of variation about the mean as the population as a whole

Random, systematic and stratified sampling

• In random sampling, members of the population have an equal chance of being selected.
• In systematic sampling, members of a population are selected at regular intervals.
• In stratified sampling, the population is divided into categories that are then sampled proportionally.

 (d) Reliability

• Variation in experimental results may be due to the reliability of measurement methods and/or inherent variation in the specimens

The precision and accuracy of repeated measurements

• The reliability of measuring instruments or procedures can be determined by repeated measurements or readings of an individual datum point.
• The variation observed indicates the precision of the measurement instrument or procedure but not necessarily its accuracy.
• The natural variation in the biological material being used can be determined by measuring a sample of individuals from the population
• The mean of these repeated measurements will give an indication of the true value being measured
• The range of values is a measure of the extent of variation in the results
• If there is a narrow range then the variation is low
• Independent replication should be carried out to produce independent data sets
• These independent data sets should be compared to determine the reliability of the results
• Overall results can only be considered reliable if they can be achieved consistently.

(e) Presentation of data

• Discrete and continuous variables give rise to qualitative, quantitative, or ranked data
• Qualitative data is subjective and descriptive.
• Quantitative data can be measured objectively, usually with a numerical value.
• Ranked data refers to the data transformation in which numerical values are replaced by their rank when the data are sorted from lowest to highest.
• The type of variable being investigated has consequences for any graphical display or statistical tests that may be used

Identification and calculation of mean, median and mode

• Use of box plots to show variation within and between data sets
• Median, lower quartile, upper quartile and inter-quartile range.
• Interpret error bars on graphical data
• Correlation exists if there is a relationship between two variables
• Correlation is an association and does not imply causation.
• Causation exists if the changes in the values of the independent variable are known to cause changes to the value of the dependent variable

Positive and negative correlations

• A positive correlation exists when an increase in one variable is accompanied by an increase in the other variable.
• A negative correlation exists when an increase in one variable is accompanied by a decrease in the other variable.

Strong and weak correlations

• Strength of correlation is proportional to spread of values from line of best fit.