Introduction to Statistical Power

Context

On numerous occasions related to decision making process we have to conclude to a particular decision, often in the form of a "yes" or "no". Say for example, we want to know whether a particular educational program is helpful for a group of students or not, so that we can decide whether that program can be applied later or not.

To decide that, we first try to measure the effect of the program on students' performance, by means of atatistical tests. Suppose an effect is found. The question arises as to - how good is the test in finding the effect, was the effect so measured significant or not? On the other hand if no effect is found - then also we try to find out whether the test was well enough to find the effect if there was any? To answer the above questions we refer to Statistical Power

Statistical power tells us, in probability terms, the capability of a test to detect a significant effect. That is, it tells us - how often can we reach a correct interpretation about the effect, if we would be able to repeat the test many times.

So power can assume values between 0 and 1 (Since probability values are expressed by numbers between 0 and 1 only). Sometimes it is expressed as a percentage - 0 referring to 0 %, and 1 referring to 100 %.

COMPONENTS Involved in Power Calculation

There are four major components which help us in the decision making process. They are:

Sample Size-

Number of study units(Students available for our study)

Effect Size-

The magnitude of the trend (How much is the post program effect discernible from the status of students before the program was instuted.

Alpha Level-

Odds of concluding the presence of an effect (when in reality there is none) due to chance only.

Power-

Odds of finding the presence of an effect when there is truly one such.

Now before we go deeper into the discussion of the above components we might be interested in knowing - "After all, why do we need to know about power."

A Priori Analysis

Let us think about the example of students and educational program. To perform the test we need to take a sample of a specified size (that is a specified number of students), we need to define how much improvement in students performance is considered to be an effect (the desired minimal detectable effect size), the estimated variance in the performance of students which might confuse us (noise). We often try to overcome these by increasing sample size, within the constraints of time and money without really being sure about how much is it going to help us! Calculation of power helps us in determining exactly how much of our effort are to be divided among obtaining a specified sample size, and other areas relavant to data analysis which also are resource-dependent.

Whereas a priori analysis helps us ensure that we don't waste valuable resources in terms of time and money on sampling subjects, we can also do some analysis after the study has been done without the power calculation done at the beginning.

Post Hoc Analysis

When we have already done the study with a specific sample size we can still evaluate our result by calculating power to find the extent of significance in our result. We will discuss more about it later.

The following Factors help us in increasing the power of a study

1. Increased time - (The total duration of study). We can't spend beyond a certain period of time. Also it so happens that we could better utilize a part of the time for better analysis rather than wasting it on not so productive data collection.
2. Increased no. of plots in the sampling frame 3. Increased no. of counts per sampling period
Both of the above involve increased effort/resources. 4. Bigger Effect Size - Salience of the (program) effect relative to the noise (Normal variations in measurements of students performance)
5. Increased Type I error- alpha level- to be found in the links.

Factors which result in decreased power

1. Increased Type II error- to be found in the links.
2.Count variance over space and time-Increased variance can mask the effect if it is small. 3.Tailednessif we want to find whether the program was helpful versus whether the program had an effect either deteriorating or beneficial for the students.

Concept Formation

Now let us develop the basic concepts through some pictorial representation

UNDER CONSTRUCTION

Some links follows

This is a good site having links to other sites.
A Primer
Some common pitfalls in power analysis - a study.
Different Look