# group project

broaden and fulfill based on the proposal to final. Attached is the part that I am responsible for.

Sample Size and Statistical PowerProvide a clear indication of the study’s sample size and the reasons for selecting this size. If the sample size may be freely chosen by the organization, then a calculation of the minimal sample size to achieve the suggested effect size based on considerations of statistical power should be performed. If there are reasons why the organization cannot select a sample size, then a calculation of the statistical power for the chosen sample size should be performed. If the design of your study does not match a setting discussed in class, then you may utilize your simulations in Part 2 to estimate the statistical power for your selected sample size.For studies with more than one outcome that will be generated together, the sample size may be selected by designating a primary outcome corresponding to the most highly prioritized research question.

Research Design: Lecture 9

Outline

Part 1: Statistical Power

Part 2: Sample Size Calculations

Part 3: Simulations

Part 1: Statistical Power

How can we know if an experiment is well designed?

Are we likely to demonstrate an effect?

How can we quantify the power of an experiment?

False Positives, False Negatives

False Positives

Hypothesis tests are usually designed to limit the likelihood of a False Positive (Type I Error) to no more than the significance

level α when the null hypothesis H0 is true.

False positives are typically considered worse mistakes than false negatives.

False positives can correspond to poor outcomes like jailing an innocent person or providing cancer treatments

to a healthy patient.

Proving an Effect

When H0 is false, there is a real effect that takes place – e.g. a relationship between the independent and dependent variables.

Rejecting H0 would conclude that the effect is reasonably supported by the data.

When H0 is false, rejecting H0 would draw the correct conclusion.

The Effect Size

The effect size is the amount of change in the dependent variable that is associated with changes in the independent variable.

This effect size is stated in terms of the parameters used in the hypotheses. For instance, in the two-sample t test, our null

hypothesis is:

H0 : μT − μC = 0

for treatment group T and control group C .

In this case, the actual difference in means would be the effect size associated with the treatment.

In other settings, the effect size could be a regression coefficient, a percentage improvement, a degree of

dissimilarity, or any number of other measures.

Effects Sizes and Interpretations

The effect size tells you the impact of a course of action that changes the independent variable.

An effect size needs to be large enough to be considered meaningful in the domain.

Demonstrating a large effect size can state the case for taking a course of action.

The effect size is an important consideration that goes beyond statistical significance. In practice, the best studies will

achieve both.

Examples: Effect Size

An additional dollar of spending on online advertising is associated with 1.43 dollars of additional revenue.

A new medication will increase a patient’s 5-year probability of survival by 10%.

Adding a video series to an educational curriculum will lead to a 3% improvement in quiz scores.

Statistical Power

When H0 is false, the probability of a false negative is β .

The power of a stastical test is the probability of rejecting H0 when it is false.

The equation for the power of a test is:

Power = 1 − β

A Framework for Calculating the Power

Select an effect size.

Assuming the effect size is true, compute the probability of rejecting H0 . This is the power for the test.

The resulting power will apply to any effect size at least as extreme as the selected effect size.

Example: A Normal Z Test

H0 : μ = 0 versus HA : μ > 0.

We will assume μ = 1 is the real effect size.

Likewise, we will assume that the standard deviation of the data is σ = 2 .

With a sample size of n = 50, what is the statistical power of the test with significance level α = 0.05?

The Power Calculation

The test statistic is:

X̄ −0

Z = σ/

.

n

√

With a one-sided test and α = 0.05, we will reject H0 if Z > z1−α , where z0.95 = 1.645 , the 95th percentile of the standard

Normal Distribution corresponding to α = 0.05.

Filling in what we know, we then say that we’ll reject H0 if:

X̄ −0

X̄

Z = σ/

=

> z1−α

n

2/ 50

√

√

We can then rearrange this equality to reject H0 if:

X̄ > ( σn ) z1−α =

√

2

1.645 = 0.465

50

√

We’ll continue this calculation on the next slide.

Solving for the Power

We need to determine:

P(X̄ > 0.465) .

By the Central Limit Theorem, X̄ ≈ N (𝚖𝚎𝚊𝚗 = 1, 𝚜𝚍 =

2

.

√50 )

Note that the mean of X̄ is based on the assumed effect size, while the standard error of X̄ is based on the assumed standard

deviation and sample size.

Then we can calculate this quantity directly in R:

pnorm(q = 0.465, mean = 1, sd = 2/sqrt(50), lower.tail = FALSE)

[1] 0.9707219

This is the statistical power of the test.

What About Smaller Effect Sizes?

μ = 0.5:

pnorm(q = 0.465, mean = 0.5, sd = 2/sqrt(50), lower.tail = FALSE)

[1] 0.5492409

μ = 0.25 :

pnorm(q = 0.465, mean = 0.25, sd = 2/sqrt(50), lower.tail = FALSE)

[1] 0.2235855

μ = 0.1:

pnorm(q = 0.465, mean = 0.1, sd = 2/sqrt(50), lower.tail = FALSE)

[1] 0.09844378

The Power Depends on the Effect Size

It is easier to detect larger effects.

If the other parameters are held constant, smaller effect sizes will have less statistical power than larger effect sizes would have.

Proving a very small effect can become quite difficult.

The Power Also Depends on the Sample Size

The standard error of X̄ is

σ

.

√n

The difference between the true effect size and the assumed value in the null hypothesis (e.g. zero) can be written in terms of

the number of standard errors.

As n grows larger, this standard error shrinks.

As n grows larger, the true effect becomes a larger number of standard errors away from the null value.

Therefore, the likelihood of rejecting H0 when it is false (the power) increases as n increases.

Part 2: Sample Size Calculations

How can we establish an appropriate sample size for a study?

What kind of assumptions do we have to make?

How do these calculations change with more complicated statistical tests?

Calculating a Sample Size

We previously assumed a sample size in order to calculate the power of a test.

We could just as easily assume a level of power in order to calculate the sample size.

However, we still have to assume an effect size and standard deviation to make this calculation.

Calculating the Sample Size: Framework

Returning to our earlier equation, we had:

X̄ −0

Z = σ/

> z1−α

n

√

and therefore:

X−0

P(Z > z1−α) = P ( σ/

> z1−α).

n

√

We will continue this calculation on the next slide.

Then we can subtract zero in a creative way, with μA being the assumed effect size:

X−μA +μA

X−0

P ( σ/ n > z1−α) = P ( σ/ n

√

√

> z1−α)

X−μA

μA

X−μA

= P ( σ/ n + σ/ n > z1−α) = P ( σ/ n

√

√

√

μA

μA

μA

> z1−α − σ/ n ) = P (W > z1−α − σ/ n ) = 1 − P (W ≤ z1−α − σ/ n ) ,

√

√

√

where W is a standard Normal random variable with mean 0 and standard deviation 1. We’ll continue this calculation on the next slide.

Calculating the Sample Size: Power Assumptions

Now we can assume a value for the power as being 1 − β, which could be a value like 0.9. Then:

μA

1 − β = 1 − P (W ≤ z1−α − σ/ n ), so

√

μA

β = P (W ≤ z1−α − σ/ n )

√

The inverse Normal function Φ−1 converts a standard Normal probability P(W ≤ k) into a quantile value. For example,

Φ−1 (0.1) = −1.28 .

With this in mind, we can solve the above equation for n :

Φ−1 (β) = Φ−1 (P (W ≤ z1−α − σ/ A n )) = z1−α − σ/ A n .

μ

μ

√

√

Rearranging this to solve for n , we have:

2

z1−α −Φ−1 (β)

n = ( μ /σ )

A

Plugging in the Values

z1−α −Φ−1 (β) 2

1.645−(−1.28) 2

n = ( μ /σ ) = (

= 34.22

)

1/2

A

In practice, this means we would need 35 sampled records.

What this means: We would have a 90% power for a test to detect a true effect size of 1 if at least 35 subjects are sampled

when the standard deviation is 2.

Assumptions

We are assuming an effect size in a power or sample size calculation. In practice, this is the minimal effect size that would

constitute a meaningful effect.

We also have to make assumptions about additional parameters such as the standard deviation of the data. This can impact the

results, too.

Then, by assuming either a sample size or a level of power, we can solve for the other quantity.

R’s pwr Library

The pwr library has standardized the calculations of sample sizes for a number of statistical tests. This includes calculations for the

following tests:

Proportions (1 and 2 sample);

t Tests (1 and 2 sample);

Correlation Test;

Chi Squared Test;

ANOVA.

Example: Power for a t Test

Calculating the power in a two-sample t test:

library(pwr)

pwr.t2n.test(n1 = 50, n2 = 35, d = 0.5, sig.level = 0.05,

alternative = “greater”)

t test power calculation

n1 = 50

n2 = 35

d = 0.5

sig.level = 0.05

power = 0.7275115

alternative = greater

Example: Sample Size for a t Test

Here we are calculating the sample size for one of the two groups:

library(pwr)

pwr.t2n.test(n1 = 50, d = 0.5, sig.level = 0.05, power = 0.9,

alternative = “greater”)

t test power calculation

n1 = 50

n2 = 111.7928

d = 0.5

sig.level = 0.05

power = 0.9

alternative = greater

With this method, any single parameter can be solved for by specifying all of the others.

Sensitivity Analyses

What would happen if your assumptions about the effect size, power, or the other parameters are not correct?

It can be useful to run sensitivity analyses that explore a range of values for each parameter.

This can help you to determine how robust your study’s design might be – and what you may need to plan for if your

assumptions don’t fully account for the real setting.

Why Calculate a Sample Size?

We want to design our experiment to have a reasonable chance of demonstrating the intended effect if it exists.

We also don’t want to needlessly add subjects to the study, especially when doing so would be costly in time and

resources.

These calculations ask us to directly address the assumptions about the effect size and other parameters.

All of these considerations intertwine the technical and strategic elements of pursuing the research.

It’s About Planning

We do all of this analysis before conducting the research study.

This planning helps us to anticipate the limitations and design flaws of a study.

Better yet, we can make adjustments to the plans before we invest in the actual work.

Part 3: Simulations

What might the actual data look like once the study is conducted?

How can we create a simulation for the experiment?

How can simulation methods help us in planning studies?

Hypothetical Data

Before you run the experiment, you don’t really know what values of the data will result.

Having an example of what the data might look like can help you create your plans for analysis.

A simulation is a method of using random number generation and scenario planning to create hypothetical data.

Example Data

Does a medication reduce average blood pressure? A randomized controlled trial was conducted.

The data for this trial might look like this:

Show

10

entries

Search:

Group

1

Treatment

BP

160.2

2

Control

164.5

3

Treatment

148.5

4

Treatment

146

5

Control

166

6

Treatment

163.9

7

Control

156.2

8

Control

169.7

9

Control

171.4

10

Control

147.9

Showing 1 to 10 of 10 entries

Previous

1

Next

Plan for Simulation

We’ll set a sample size for the experiment.

We’ll randomly assign subjects to the treatment and control groups.

We’ll assume an effect size.

We’ll make other assumptions about the distribution of blood pressure (Normal) and the other parameters (means, standard

deviations, and others as needed).

Example Simulation

n

## We've got everything to become your favourite writing service

### Money back guarantee

Your money is safe. Even if we fail to satisfy your expectations, you can always request a refund and get your money back.

### Confidentiality

We don’t share your private information with anyone. What happens on our website stays on our website.

### Our service is legit

We provide you with a sample paper on the topic you need, and this kind of academic assistance is perfectly legitimate.

### Get a plagiarism-free paper

We check every paper with our plagiarism-detection software, so you get a unique paper written for your particular purposes.

### We can help with urgent tasks

Need a paper tomorrow? We can write it even while you’re sleeping. Place an order now and get your paper in 8 hours.

### Pay a fair price

Our prices depend on urgency. If you want a cheap essay, place your order in advance. Our prices start from $11 per page.