Last week I made it my goal to work through KA for 1.5 hours everyday from Tuesday to Saturday. I only managed to do it on Tuesday. The other four days I did get roughly 1.25 hours each day, so overall it wasn’t too bad and better than the 1 hour minimum I used to give myself. I also ended up doing close to 2 hours on Sunday which certainly put me over the 7.5 hours total for the week I was aiming for.

I made it through the unit Confidence Levels on Sunday morning but didn’t manage to get started on the next unit after that. What’s worse is I don’t feel like I have a very strong understanding of what I learned this week as it is. I passed the unit test with 100% score on the first try but I wasn’t confident going through it. In terms of my progress through KA, for the most part, I wasn’t too thrilled with how this week. Something random happened this week, however, that was the most exciting thing to happen to me in a long time. (I’ll explain below.)

As the title of the unit would suggest, I spent this entire week learning about what confidence intervals are and how they work. I briefly started this unit in Week 56 but only through 2 or 3 videos which explained that a confidence interval allows you to come up with a confidence level which means, “you can expect x% of y intervals to contain the parameter of interest”. In layman’s terms, it means that if you took a normally distributed sample from a population and ran it through a confidence interval equation, you could determine that the sample has x% chance of containing the population mean. The confidence interval formula for a Z- or a T-statistic is:

• Con. In. = x̅ (+/-) Z^* (or T^*) * (σ (or S)/√n)

When it comes to calculating confidence intervals using Z- and T-tables (explained below), they can only be calculated if 3 conditions are met:

1. Random Sample Condition
   • The sample taken form the population must be chosen at random. Often the questions I’m given in KA will state “an SRS (simple random sample) was taken of…” or “a random sample was taken of…” to make it clear that the sample was chosen at random.
2. Normal Condition
   • You must be able to infer that the sample distribution follows a normal distribution. This can be assumed if:
     • The parent distribution, i.e. the population, follows a normal curve, or
     • For a Z-table, if there are ≥ 10 ‘successes’ AND ≥ 10 ‘failures’, or
     • For a T-table, n ≥ 30.
3. Independence Condition
   • The sample taken must be an independent sample, i.e. selecting/making an individual observation has no effect on the probability of selecting/making another individual observation for the sample. You can infer this if:
     • Each observation is taken and then replaced back into the population before selecting again, or
     • The sample taken is ≤ 10% of the population (i.e. the 10% Rule).

Writing this post, I realize I really don’t understand much about the difference between a Z-table/score and a T-table/score. I don’t understand how or why the formulas work. Here’s a page from my notes that somewhat explains the difference:

I think the most important takeaway from these notes is that to calculate the Z- or T-score from a sample mean (x̅) you need to use the formula:

• Z_ x̅ or T_ x̅ = (x̅ – μ x̅)/(S/√n)
• And use a Z-table when S ≥ 30 and a T-Table when S < 30.

I feel like I’m very close to understanding all of this but I’m not quite there. For me, if I understand 95% of a concept I might as well understand 0% of it. I need to wrap my head around every single aspect of the concept and understand 100% of it and suddenly a lightbulb goes off, everything falls into place, and the entire thing becomes completely obvious. I have a feeling I’m on the brink of having that happen with Z-table/scores and T-tables/scores but I’m not quite there and it’s SUPER frustrating!!! :@

Here are a list of new terms/definitions I picked up this week:

• Standard Error 
  • Notation = σ_p̂
  • To find the S.D. of a Bernoulli distribution you use the formula:
    • σ_ X = √ (P * (1 – P)/N)
  • However, when finding the S.D. of a sample, i.e. σ_p̂, we can use the same formula but replace P with p̂ to get:
    • σ_ p̂ = √ (p̂ * (1 – p̂)/n)
  • BUT we then refer to σ_ p̂ as the standard error instead of the S.D.
    
• Critical Value
  • Notation = Z^* or T^* (pronounced “Z star” and “t star” respectively)
  • Not sure exactly what this is, but it seems to be another term for standard deviation.
  • Used to calculate Margin of Error.
    
• Degree of Freedom
  • Notation = n – 1 
  • This is something I REALLY don’t understand. Here’s a very simple analogy I found from a website that gives an ELI5 version of what the degree of freedom is:
    • “Imagine you’re a fun-loving person who loves to wear hats. You couldn’t care less what a degree of freedom is. You believe that variety is the spice of life. Unfortunately, you have constraints. You have only 7 hats. Yet you want to wear a different hat every day of the week. On the first day, you can wear any of the 7 hats. On the second day, you can choose from the 6 remaining hats, on day 3 you can choose from 5 hats, and so on. When day 6 rolls around, you still have a choice between 2 hats that you haven’t worn yet that week. But after you choose your hat for day 6, you have no choice for the hat that you wear on Day 7. You must wear the one remaining hat. You had 7-1 = 6 days of “hat” freedom—in which the hat you wore could vary!” – blog.minitab.com
      
• Margin of Error
  • Tells you how many percentage points your results will differ from the mean in a sample from the real population value.
  • It is a range of values below and above the sample statistic in a confidence interval.
  • For example, a 95% confidence interval with a 4% margin of error means that your statistic will be within 4% of the real population value 95% of the time.
  • The Margin of Error formula can be found within the Confidence Interval formula and is the section:
    • Z^* (or T^*) * (σ (or S)/√n) = Margin of Error
      
• Tail probability
  • The probability of the tails on either a normal distribution or a T-distribution being empty.

One thing I did get a better this week was understand the difference between a sample mean (x̅) and a sample parameter (p̂). Here’s a table I found online that gives some good detail on the two and their differences (not sure what the bottom right section is talking about):

In essence, a sample mean is found by adding up all the individual observations from a sample and dividing by the number of observations to get the mean of the sample, whereas a sample parameter is found when taking a sample from a Bernoulli distribution and finding the mean between the successes (1) and failures (0).

Even though I struggled with a lot of things this week, there were still a few bright spots:

1. First, although most of the questions I was given that required me to use a S.D. or standard error gave me the value each right off the bat, I sometimes calculated them on my own and was able to do it perfectly every time and fully understood how to do it without needing to review. 
   
2. Second, when calculating the margin of error in some questions, Sal said something along the lines of, “we don’t typically use algebra this advanced at this level of statistics so don’t worry if you don’t understand it” and yet I found the algebra we used super easy. 
   
3. Third, I finally completely figured out how ‘flipping inequalities’ when taking their reciprocals and when multiplying them by a negative number. I did this by thinking of simple example inequalities and going through both types of transformations. For example:

• Reciprocals:
  • 3/4 > 2/4  
    • = 0.75 > 0.5
  • 4/3 < 4/2 
    • = 1.33 < 2
      
• Multiplying by a negative:
  • 3/4 > 2/4 
    • = 0.75 > 0.5
  • -3/4 < -2/4 
    • = -0.75 < -0.5

It seems unlikely that I’ll get through all the stats courses by the end of October. A more realistic goal might be to aim to get through this course by the end of October and the other two courses by the end of November. I’m a broken record by saying this, but I really want to understand the material well before moving on. I think it’s likely that it’s going to take me a bit of time to do that. My goals for this coming week is to get halfway through the new unit Significance Tests (Hypothesis Testing) (0/1500 M.P.). After I get through this unit, I’ll still have 4 more to go, however only one of them actually has Mastery Points (which seems weird) and it only has 700 M.P. which is relatively small. Considering how long I’ve been working on through this course and how much of a struggle it has been at times, it’s a bit crazy to see the end of it coming up!

Lastly, as I mentioned in my intro, one of the most exciting things to happen to me in a long time happened this past Friday, October 2^nd. I was taking my dog for a walk in High Park and ran into a hero of mine, Jordan Peterson. Jordan’s had a strong influence on me over the past two years and has probably been the main reason why I started doing KA and why I’ve been pushing myself to take on more responsibility and improve my life, in general. I’ve quite literally watched hundreds of hours of his interviews and lectures on YouTube and hearing the advice he gives has really, really helped make my life better.

I ran into him on the main off-leash path in the park. Jordan has been having a lot of health issues lately and I could tell that his wife Tammy was helping him walk. My jaw literally dropped when I saw him and he could clearly tell that I recognized him. He said hello and, seeing that he was having a hard time walking and knowing about his health issues, I asked him, “are you ok?”. He nodded and said he was doing ok. This may sound ridiculous but I started to cry. I looked down to the ground to hide the fact that I was crying and when I looked up he had walked over with his hand out and asked me what my name was. I told him my name was Will and asked him if I could give him a hug, which I did. He was tearing up at that point, too. I told him I didn’t want to take up too much of his time and thanked him and told him I watched his videos for 2 hours every day (which is not accurate but I wasn’t thinking properly in the moment). I don’t remember what he said or if he even said anything at all after that. I let him go and watched him and his wife walk away.

I probably sound like a teenage fangirl saying this, but meeting him and having him acknowledge me and give me a hug was one of the greatest feelings I’ve had in a long, long time. It’s motivated me even more to continue to improve myself and hopefully help improve the world for others.