This chapter talks about the performance of a simulation analysis in the contexts of the precision of a parameter estimate, Bayesian statistics, and model selection. It describes null hypothesis testing as an absolute dominant form of statistical inquiry that remains important to a lot of researchers in many different areas of the life sciences. It also reviews toolkits of statistical approaches which are in common use alongside null hypothesis testing. The chapter reviews the simulation approach to power in the context of null hypothesis testing and other contexts that estimate the size of a parameter or select from a number of alternative models. It uses examples to explore simulation analysis in the context of precision of parameter estimation.
12
Chapter
Applying our simulation approach beyond null hypothesis testing: parameter estimation, bayesian, and model-selection contexts
Chapter
Between-individual variation, replication, and sampling
This chapter explores the purpose of experimental design, which is about removing or controlling for variation due to factors that are not of interest in order to see the effects of factors that are of interest. It talks about measuring different experimental subjects rather than just a single individual as a key aspect to coping with variation. It also demonstrates the identification of a population of independent experimental subjects and its sampling in several different ways. The chapter discusses the terms random variation, inherent variation, background variation, extraneous variation, within treatment variation, and noise, which describes any variation in the response variable between-individuals in a sample that cannot be attributed to the independent factors. It explains replication as a way of dealing with the between-individual variation due to the random variation that will be present in any life science situation.
Chapter
Beyond complete randomization: blocking and covariates
This chapter explains the concept of blocking in experimental design and explores the range of things that would be interesting to be blocked on. It looks at the potential costs of blocking, which can help decide whether it is an attractive design for a given experiment. It also considers paired designs as a special form of blocking with only two subjects in a block and talks about the best way select the size of blocks. The chapter introduces covariates and discusses interactions involving covariates. It highlights the inclusion of factors that vary continuously as a covariate in a study, which can be a viable alternative to blocking if the trait in question is continuous and has a linear effect on the trait measured in a study.
Chapter
Dealing with multiple hypotheses
This chapter explains how to evaluate the powers for study designs with multiple hypotheses, emphasizing that no single design will offer exactly the same power for all the hypotheses. It points out that ranking hypotheses in terms of their importance can help with the process of finding a design that offers sufficient power for all the hypotheses of prime importance. It also discusses how to evaluate whether the power available for testing ancillary hypotheses is sufficient to continue to include them in the study after one has settled on a design with sufficient power for the most important hypotheses. The chapter explores how to select a single experiment offering good power to address all hypotheses and involve further simulations after carrying out power analyses for each of the hypotheses in isolation. It examines power analyses that flag up situations where bonus investigations are of little value. This is because an experiment optimized to address questions of primary interest means there is low power to address bonus hypotheses.
Book
Edited by Graeme D. Ruxton and Nick Colegrave
Experimental Design for the Life Sciences starts off by asking why we should care about design. It then looks to how to start with a well-defined hypothesis. The text covers questions involved in selecting the broad design of a study. It looks at between-individual variation, replication, and sampling. There are also chapters on pseudoreplication, sample size, power, effective design, experimental design, and factorial designs. Finally, the text considers beyond complete randomization, within-subject designs, and measurements.
Chapter
Experiments with several factors (factorial designs)
This chapter reviews the concept and terminology associated with the extension of the randomized design and discusses the concept of interaction between multiple factors. It introduces split-plot and Latin square designs and explains their attractions and limitations. It also explains how split-plot designs are used on experiments with multiple factors where one-factor can be allocated easily to groups of experimental units but finds it difficult to assign to individual experimental units. The chapter describes the Latin square design as an alternative to a fully randomized factorial design that uses fewer experimental units. It clarifies how certain design concepts link to statistical analysis.
Chapter
Extensions to other designs
This chapter covers how to simulate experimental studies where the predictor variable is continuous and there is a straight-line relationship between the response and the predictor. Moreover, it explores simulations that include other kinds of relationships, such as a quadratic one. It also demonstrates how to deal with response variables which are not continuous, as well as with variation that does not follow a normal distribution. The chapter looks at another common type of biological study in which both the response and the predictor variable are continuous. It talks about the possibility of having a different number of replicates at each level and explores situations where the relationship is either linear or non-linear.
Chapter
How to quantify power by simulation
This chapter focuses on the necessity of being able to make estimates concerning a potential study design to assess its power. It explains how to generate assumptions and convert them into a good estimate of statistical power by conducting a simulation of a potential study in R or in any other programming language. The chapter also shows how to modify one’s approach to compare multiple possible versions of a study to decide which one offers the power that is needed most efficiently. The key aspects involved in estimating power covered in the chapter can be applied to any study. The chapter also introduces basic computational skills which form part of the toolkit that is needed to perform power analyses.
Chapter
Hypothesis testing: Associations and relationships
This chapter takes a look at the second hypothesis that may be tested: a test for an association. An association is where one variable changes in a consistent and similar manner to another variable. There are three groups of statistical tests that can be used: chi-squared tests for association, correlation analysis, and regression analysis. The chapter discusses what is meant by an association and a relationship, and issues relating to limited experimental designs and small sample sizes. Each test is illustrated with an example from undergraduate research. The statistics tests covered in the chapter are: chi-squared test for association; Spearman's rank correlation; Pearson's product moment correlation; coefficient of determination; simple linear regression; linear regression; principal axis regression; and ranged principal axis regression.
Chapter
Hypothesis testing: Do my data fit an expected ratio?
This chapter considers statistical tests and experimental designs that will be suitable for testing the hypothesis, ‘Do my data fit an expected ratio?’ The statistical test most often used to test this hypothesis is the chi-squared goodness-of-fit test. The chapter explains more about this type of hypothesis, how to carry out the necessary calculations, and how to resolve some of the problems that may be encountered. The statistics tests covered in the chapter are: chi-squared goodness-of-fit test: one sample. It then explores how to check whether the data have a normal distribution using the chi-squared goodness-of-fit test. Finally, the chapter takes a look at replication in a goodness-of-fit test.
Chapter
Hypothesis testing: Do my samples come from the same population? Non-parametric data
This chapter discusses how to test the hypothesis that samples come from the same statistical population using non-parametric tests. It includes the tests used most often by undergraduates and which mirror those covered in the previous chapter for testing the same hypothesis with parametric data. There are a number of key principles that need to be understood in relation to the use and interpretation of these tests, including how to rank the data, absolute values, matched data, and the interaction between treatment variables. These points are covered when they first become relevant to the statistics tests being considered. Additionally, these tests are ‘distribution free’ and the data therefore do not have to be distributed normally.
Chapter
Hypothesis testing: Do my samples come from the same population? Parametric data
This chapter talks about how we may analyse parametric data to test the hypothesis that two or more samples come from the same statistical population. These data are usually presented in tables or summary histograms rather than a scatter plot, and the treatment variables are usually measured on an ordinal and/or nominal scale, or occasionally measured on an interval scale and arranged into discrete categories. Other (dependent) variables are measured on an interval scale. All these tests use the underlying mathematical relationships that come from the data having a normal distribution. The tests outlined in the chapter cover a wide range of experimental designs with one or more treatment variables, with or without replicates. Particular design features are covered in the relevant sections; however, whether observations are matched or unmatched is a central point common to all these tests.
Chapter
Improving the power of an experiment
This chapter discusses why inherent variation is a challenge to achieving good statistical power and how design decisions can reduce or control the effects of inherent variation. It demonstrates the increase of power by changing the experimental design to increase the strength of the effect that should be detected. It also addresses why increasing sample size increases power and why higher power will be more costly to achieve. The chapter talks about changes to the experiment that increase its power but may incur other costs, which could be felt financially, in terms of increased effort required, or could involve ethical issues. It explores the quantification of power benefits and the costs of different options for an experiment that have a rational basis for choosing the alternative with the most attractive trade-off between power and costs.
Chapter
An introduction to hypothesis testing
This chapter considers the general principles behind hypothesis testing. There are six steps to follow in all hypothesis testing. The first is to confirm which distribution is the best one to use. Next, the chapter shows how to select and correctly phrase the hypotheses to be tested and determine which is/are the correct statistical test(s) to use. Afterwards, the calculated value of the test statistic must be worked out. The critical value of the test statistic is then found. The chapter also takes a look at whether or not to reject the null hypothesis depending on the rule relating to the critical and calculated test statistics. Finally, the chapter considers how to decide what the outcome of the statistical test means in terms of the biology of the system.
Chapter
INTRODUCTION: WHY SHOULD YOU READ THIS BOOK?
Part 1: Why you should want to do power analysis
If we wanted to persuade you to read this book, we could try and scare you. We could say that your career as a scientist is really going to be held back if you don’t...
Chapter
Planning your experiment
This chapter goes through the process of planning experiments. The best way to learn about the design of investigations is to look at what other people have done and make a similar attempt. The chapter shows how to understand the strengths and weaknesses of an experimental design and how to identify the ways in which the design might be improved. It also provides information about the process of designing experiments and the things we would need to consider. To that end, the chapter considers how to critically review published experimental designs. The first step in this process is to identify the aim and objectives and then to consider the strengths and weaknesses of the work. Next, the chapter explores the process of designing experiments using the decision web.
Book
Nick Colegrave and Graeme D. Ruxton
Power Analysis starts by asking: what is statistical power and why is low power undesirable? It then moves on to considering ways in which we can improve the power of an experiment. It asks how we can quantify power by simulation. It also examines simple factorial designs and extensions to other designs. Next, it asks how we can deal with multiple hypotheses. Finally, it looks at how to apply the simulation approach presented in this book beyond null hypothesis testing.
Chapter
Pseudoreplication
This chapter focuses on pseudoreplication, in which subjects are not independent of each other but must be treated as if they were. It discusses important details of what is meant by independence and outlines common sources of non-independence. It also elaborates how non-independence is inevitable in certain situations and points out how it can invalidate a study if not handled carefully. The chapter illustrates pseudoreplication that can be linked to the concepts of third variables and confounding factors and looks at issues of non-independence in studies that involve looking for change over time. It emphasizes that the problems of third variables in correlation studies can equally be thought of as a problem of pseudoreplication and confounding factors.
Chapter
Questionnaires, focus groups, and interviews
This chapter provides a brief introduction to the use of questionnaires, focus groups, and interviews to gather scientific information. It discusses the design of the questions in relation to the sample size and how the data may be analysed quantitatively. Questionnaires, focus groups, and interviews may allow us to collect both quantitative (numerical) data and qualitative (descriptive) information. Qualitative approaches are based on the notion that reality varies for different people in different contexts; therefore, we cannot use a single scale against which to make measurements. An example of this is people's perception of pain. Qualitative information can be summarized, but here the focus is on how these methods may be used effectively to obtain numerical or quantitative data and how these data may be analysed.
Chapter
Reporting your research
This chapter goes through the most common approaches taken to communicating findings to research. This includes: writing a report, presenting a poster, and giving a presentation. These all have similar elements: Title, Introduction, Methods, Results, and Discussion. The chapter thus explains what each section is for, approaches to take when writing each section, common errors, and issues specific to each form of communication. It first provides a guide to the general format common in scientific papers and reports. The chapter then demonstrates how, in a research poster, the written content and the visual tools used in the presentation are of equal importance. Finally, using oral presentation to discuss results is covered.
12
