The Power Hypercube: A tool for exploring the concept of Power

These are a series of screen shots from the software program G*Power meant to illustrate the basic trade-offs in power analysis, e.g., sample size and target effect size versus power, and Type I (level) versus Type II (power) error. To keep things simple, only one of the simplest tests is considered: The one-sample t-test of the sample mean against a constant value (which we assume is zero).

The way they work is fairly simple, each of the four links below leads to a screen shot from G*Power. Along size is a list of possible values for the various values you can manipulate:

Level (Type I error rate)
The chance of spuriously rejecting the null hypothesis when it is true.
Power (1-Type II error rate)
The chance of correctly rejecting the null hypothesis when the true population mean is given by the effect size.
Sample Size (N)
The size of the sample, a simple random sample is assumed.
Effect Size (Cohen's d)
How much the specific alternative hypothesis for which the power is calculated differs from the null hypothesis. For the one-sample t-test, this is the difference (in standard deviations) between the population mean and the mean under the null hypothesis. By convention, d=0.2 is considered small, d=0.5 is considered moderate and d=0.8 is considered large, although what is considered an adequate effect size may be very dependent on the discipline.

G*Power has a number of modes in which it can operate (basically, if you supply any three of the values above, it will calculate the fourth one), two are available below:

A Priori Analysis
In these screen shots, the sample size is the value that is calculated.
Post Hoc Analysis
In these screen shots, the power is the value that is calculated.
As part of the planning process of an experiment, researchers should always conduct a power analysis to determine the size of the sample they need. According to the textbooks, the a priori analysis, which calculates the sample size to meet the research goals, is the one that should be used. In practice, the sample size is usually constrained by the budget of the project. In this case, a post hoc analysis can be used to calculate the power available for the target effect at the available sample size; if the power is adequate, then doing the experiment will be worth while. If the power is not adequate, the experiment needs to be redesigned or maybe even abandoned.

As experimental design is often a matter trade-offs, it is more helpful to look at these things graphically. G*Power also offers a way to do those graphs. Two are provided:

References

G*Power is free and available for download from its home page (linked above). It allows the calculations shown here for any sample size or effect size, and not just the few sampled for the Power Cube. It also supports a lot more tests.

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191.

This page was design and maintained by Russell Almond (ralmond@fsu.edu). Last Modified 2013-03-22.