| A | B |
|---|
| Statistically significant | When the P-value falls below the alpha level, we say that the test is this term at that alpha level. |
| Alpha level | The threshold P-value that determines when we reject a null hypothesis. If we observe a statisic whose P-value based on the null hypothesis is less than this term, we reject that null hypothesis. |
| Significance level | The alpha level is also called this, most often in a phrase such as a conclusion that a particular test is "significant at the 5% significance level." |
| Critical value | The value in the sampling distribution model of the statistic whose P-value is equal to the alpha level. A statisitc value farther from the null hypothesis value than the critical value will have a smaller P-value than alpha and will lead to rejecting the null hypothesis. This value is determined with z star. |
| Type I error | The error of rejecting a null hypthesis when in fact it is true (false positive). The probability of this error is equal to alpha. |
| Type II error | The error of failing to reject a null hypothesis when in fact it is false (false negative). The probabilltiy of this error is commonly denoted Beta and depends on the effect size. |
| Power | The probability that a hypothesis test will correctly reject a false null hpothesis is the power of the test. To find power, we must specify a particular alternative parameter value as the "true" value. For any specific value in the alternative, the power is 1 - Beta. |
| Effect size | The difference between the null hpothesis value and true value of a model parameter is called the effect size. |
