The Power Hypercube: A tool for exploring the concept of Power
These are a series of screen shots from the software program
G*Power
meant to illustrate the basic trade-offs in power analysis, e.g.,
sample size and target effect size versus power, and Type I
(level) versus Type II (power) error. To keep things simple,
only one of the simplest tests is considered: The
one-sample t-test of the sample mean against a constant value
(which we assume is zero).
The way they work is
fairly simple, each of the four links below leads to a screen shot
from G*Power. Along size is a list of possible values for the various
values you can manipulate:
- Level (Type I error rate)
- The chance of spuriously rejecting the null hypothesis when it
is true.
- Power (1-Type II error rate)
- The chance of correctly rejecting the null hypothesis when the
true population mean is given by the effect size.
- Sample Size (N)
- The size of the sample, a simple random sample is assumed.
- Effect Size (Cohen's d)
- How much the specific alternative hypothesis for which the
power is calculated differs from the null hypothesis. For the
one-sample t-test, this is the difference (in standard
deviations) between the population mean and the mean under the null
hypothesis. By convention, d=0.2 is considered
small, d=0.5 is considered moderate and d=0.8 is
considered large, although what is considered an adequate effect
size may be very dependent on the discipline.
G*Power has a number of modes in which it can operate (basically,
if you supply any three of the values above, it will calculate the
fourth one), two are available below:
- A Priori Analysis
- In these screen shots, the sample size is the value that is
calculated.
- Post Hoc Analysis
- In these screen shots, the power is the value that is
calculated.
As part of the planning process of an experiment, researchers should
always conduct a power analysis to determine the size of the sample
they need. According to the textbooks, the a priori analysis,
which calculates the sample size to meet the research goals, is the one
that should be used. In practice, the sample size is usually
constrained by the budget of the project. In this case, a post
hoc analysis can be used to calculate the power available for
the target effect at the available sample size; if the power is
adequate, then doing the experiment will be worth while. If the power
is not adequate, the experiment needs to be redesigned or maybe even
abandoned.
As experimental design is often a matter trade-offs, it is more
helpful to look at these things graphically. G*Power also offers a
way to do those graphs. Two are provided:
References
G*Power
is free and available for download from its home page (linked above).
It allows the calculations shown here for any sample size or effect
size, and not just the few sampled for the Power Cube. It also
supports a lot more tests.
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3:
A flexible statistical power analysis program for the social,
behavioral, and biomedical sciences. Behavior Research
Methods, 39, 175-191.
This page was design and maintained
by Russell Almond
(ralmond@fsu.edu). Last
Modified 2013-03-22.