This chapter talks about the performance of a simulation analysis in the contexts of the precision of a parameter estimate, Bayesian statistics, and model selection. It describes null hypothesis testing as an absolute dominant form of statistical inquiry that remains important to a lot of researchers in many different areas of the life sciences. It also reviews toolkits of statistical approaches which are in common use alongside null hypothesis testing. The chapter reviews the simulation approach to power in the context of null hypothesis testing and other contexts that estimate the size of a parameter or select from a number of alternative models. It uses examples to explore simulation analysis in the context of precision of parameter estimation.
Chapter
Applying our simulation approach beyond null hypothesis testing: parameter estimation, bayesian, and model-selection contexts
Chapter
Dealing with multiple hypotheses
This chapter explains how to evaluate the powers for study designs with multiple hypotheses, emphasizing that no single design will offer exactly the same power for all the hypotheses. It points out that ranking hypotheses in terms of their importance can help with the process of finding a design that offers sufficient power for all the hypotheses of prime importance. It also discusses how to evaluate whether the power available for testing ancillary hypotheses is sufficient to continue to include them in the study after one has settled on a design with sufficient power for the most important hypotheses. The chapter explores how to select a single experiment offering good power to address all hypotheses and involve further simulations after carrying out power analyses for each of the hypotheses in isolation. It examines power analyses that flag up situations where bonus investigations are of little value. This is because an experiment optimized to address questions of primary interest means there is low power to address bonus hypotheses.
Chapter
Extensions to other designs
This chapter covers how to simulate experimental studies where the predictor variable is continuous and there is a straight-line relationship between the response and the predictor. Moreover, it explores simulations that include other kinds of relationships, such as a quadratic one. It also demonstrates how to deal with response variables which are not continuous, as well as with variation that does not follow a normal distribution. The chapter looks at another common type of biological study in which both the response and the predictor variable are continuous. It talks about the possibility of having a different number of replicates at each level and explores situations where the relationship is either linear or non-linear.
Chapter
How to quantify power by simulation
This chapter focuses on the necessity of being able to make estimates concerning a potential study design to assess its power. It explains how to generate assumptions and convert them into a good estimate of statistical power by conducting a simulation of a potential study in R or in any other programming language. The chapter also shows how to modify one’s approach to compare multiple possible versions of a study to decide which one offers the power that is needed most efficiently. The key aspects involved in estimating power covered in the chapter can be applied to any study. The chapter also introduces basic computational skills which form part of the toolkit that is needed to perform power analyses.
Chapter
Improving the power of an experiment
This chapter discusses why inherent variation is a challenge to achieving good statistical power and how design decisions can reduce or control the effects of inherent variation. It demonstrates the increase of power by changing the experimental design to increase the strength of the effect that should be detected. It also addresses why increasing sample size increases power and why higher power will be more costly to achieve. The chapter talks about changes to the experiment that increase its power but may incur other costs, which could be felt financially, in terms of increased effort required, or could involve ethical issues. It explores the quantification of power benefits and the costs of different options for an experiment that have a rational basis for choosing the alternative with the most attractive trade-off between power and costs.
Chapter
INTRODUCTION: WHY SHOULD YOU READ THIS BOOK?
Part 1: Why you should want to do power analysis
If we wanted to persuade you to read this book, we could try and scare you. We could say that your career as a scientist is really going to be held back if you don’t...
Book
Nick Colegrave and Graeme D. Ruxton
Power Analysis starts by asking: what is statistical power and why is low power undesirable? It then moves on to considering ways in which we can improve the power of an experiment. It asks how we can quantify power by simulation. It also examines simple factorial designs and extensions to other designs. Next, it asks how we can deal with multiple hypotheses. Finally, it looks at how to apply the simulation approach presented in this book beyond null hypothesis testing.
Chapter
Simple factorial designs
This chapter elaborates on the organization of data from experiments in a spreadsheet and uses that as a guide to producing exactly analogous simulated data. It discusses the production of simulated data which is organized in the same way as actual experimental data. This makes the calculation of power for more complicated designs much easier. The chapter also demonstrates the use of power analyses to explore the effects of adverse events which lead to an unplanned loss of some data. The chapter cites hypothetical studies, such as determining the resistance to herbicides and uses two genotypes of Chlamydomonas reinhardtii, a single-celled algae that is a convenient lab system, to explore questions about herbicides.
Chapter
What is statistical power?
This chapter defines the terms 'population' and 'sample' in the context of a scientific study and identifies the null hypothesis associated with any scientific question. It determines what inferences can and cannot be drawn from a p-value and elaborates the concepts of Type I and Type II error in terms of null hypothesis testing. It also explains statistical power and analyses its link to Type II error. The chapter discusses the study of an appropriate representative sample of the population of interest and the use of statistics as an attempt to make generalizations about the population that the sample represents. It introdduces tools developed by statisticians that use information from samples to draw inferences about the populations that they represent.
Chapter
Why low power is undesirable
This chapter addresses the questions why a low-powered study has a high probability of failing to detect genuine effects. It highlights how low power makes drawing useful inferences from failure to reject the null hypothesis more difficult. It shows that the risk that a significant result is a Type I error increases if an experiment is low-powered and analyses why low power inflates the effect sizes associated with statistical significance. The chapter clarifies that the statistical power of an experiment is one minus the probability of making a Type II error based on the outcome of that experiment. It confirms that low power is undesirable because it means an increased risk of a Type II error, emphasizing that the lower the power, the less likely it is that a significant result is actually triggered by a real underlying effect.
