Surprisingly, this week went exactly according to plan. I finished 800/1500 M.P. on the unit Significance Tests (Hypothesis Testing) which was my goal and, furthermore, I actually fully understood all the new material I went through. To be fair, it hasn’t all completely sunk in and I wouldn’t necessarily be able to teach it to someone BUT nothing came up which I wasn’t able to wrap my head around. There were a number of new definitions I learned this week but it was also a good balance of working through equations. Overall this was one of the most balanced weeks that I’ve had in the 58 weeks I’ve been at it.
(I always feel a bit uneasy when things going to go according to plan. Almost like there’s no possible way things could have worked out perfectly and that I must be forgetting about something bad that happened…)
Before I explain what a significance test is, I think it’s best to first talk about what is known as a Null Hypothesis and an Alternative Hypothesis:
- Null Hypothesis
- H_o (‘o‘ stands for ‘original’ hypothesis)
- The assumed hypothesis.
- Ex. A school assumes their students are getting 8 hours of sleep per night.
- H_o: μ = 8
- Alternative Hypothesis
- H_a
- A suspected alternate hypothesis.
- “Claims about the population that you care about.” – Sal
- Ex. The school randomly selects 42 of their students and finds their mean hours of sleep to be 7.5 hours.
- H_a: x̅ = 7.5
A significance test compares the sample mean/parameter to the population mean/parameter to see how likely it was to occur. As far as I understand, a significance test in layman’s terms is when you:
- Take a sample from a normally distributed population and find the sample mean/parameter (i.e. x̅ or p̂),
- Then compare that sample mean/parameter to the original hypothesis’s mean/parameter (i.e. μ or P), and
- If the sample mean/parameter is significantly far enough away from the H_o mean/parameter, you can ‘reject’ the original hypothesis.
In order to properly conduct a significance test you must first set a Significance Level AND also understand what a P-Value is:
- Significance Level
- α (i.e. the Greek letter for “alpha”).
- A predetermined percentile that, if the sample mean/parameter is lower than, validates the possibility of rejecting the original hypothesis.
- Often set to 0.05.
- Note: Does not always need to be set to 0.05. It is often set to 0.1 or 0.01 and is at the discretion of whoever’s conducting the study.
- MUST be set before the test is conducted.
- P-Value
- Stands for ‘Probability’-Value.
- (As far as I understand) is the value taken from a Z-table using the Z-score of the sample mean/parameter.
- The area of the normal distribution toward the end of the ‘tail’ from wherever the sample mean/parameter’s location is on the distribution.
- Ex. In a one-tailed significance test, if a sample mean was located exactly –2 S.D.’s away from μ, the P-value would be equal to 0.02275 which is –2.00 on a Z-table. This means that there is a 2.275% chance of the sample mean being that far away from the population mean.
- Always compared to the significance level to determine if you can reject the original hypothesis (i.e. the P-value < α) or fail-to-reject the original hypothesis (i.e. the P-value > α).
When conducting significance tests there’s always the possibility of getting results that suggest the opposite of the truth (i.e. they suggest an inaccurate conclusion). There are two possible errors that can occur on a significance test known as Type 1 Errors and Type 2 Errors:
- Type 1 Error
- Occurs when in reality the original hypothesis is TRUE but the results from the significance test would have you conclude that the original hypothesis is FALSE and reject it.
- Occurs when the P-value from a test is less-than the significance level, meaning the chance that a Type 1 Error will occur will EQUAL the significance level.
- Ex. If significance level is 0.05 than the chance of a Type 1 Error occurring equals 5%.
- Type 2 Error
- Occurs when in reality the original hypothesis is FALSE but the results from the significance test would have you conclude that the original hypothesis is TRUE and fail-to-reject it.
- Occurs when the P-value from a test is greater-than the significance level, meaning the chance that a Type 2 Error will occur will be (1 – significance level).
- Ex. If significance level is 0.05 than the chance of a Type 2 Error occurring equals 95%.
| H_o True | H_o False | |
| Reject H_o | Type 1 Error | Correct |
| Fail-to-Reject H_o | Correct | Type 2 Error |
The last thing I learned about this week us what’s known as Power in relation to significance tests. Here is a page from my notes that explains this concept:
(Note: In the photo, “OG Hyp. means original hypothesis.)
The simplest way to conceptualize what power is would be to imagine 2 normal distributions which overlap (just like in the photo), the first being the original hypothesis’s distribution and the second being the alternative hypothesis’s distribution. If in reality the alternative hypothesis’s distribution is accurate and the original hypothesis’s distribution is inaccurate, Power is the area of the alternative distribution that doesn’t cross over the significance level and overlap onto the original distribution. (See, simple right?) Another way to think about it is that power is the probability that you DON’T make a Type 2 Error.
Though I understand how it works, I feel like I don’t fully understand why power is important. I do know, however, that there are 3 ways to increase power:
- Increase the sample size,
- Have the true parameter, H_a, be further away from the original parameter, H_o, (although that’s not in your control) and/or
- Decrease the significance level.
With there only being a few small units remaining in this course, I’m starting to feel like I’m getting pretty close to finishing off statistics! I know there are still two more courses after this but, as I’ve said before, from how much is already completed in each course, they’ll mostly be review. This coming week, Week 59, I’d like to finish off this unit Significance Tests (Hypothesis Testing) (800/1500 M.P.). The following unit, Two-Sample Inference for the Difference Between Groups (No M.P.), is only made up of 9 videos and a practice exercise so I think it may be possible to also get through that unit this coming week, as well. Either way, it’s nice to be on the home-stretch and to finally see the light at the end of the long statistics tunnel!