Statistical Machine Learning

Bayesian World View of Cognition

Basics

STAT 400: Statistics and Probability I

• Variance of a random vector $X$ along a direction $u$: $V[u^TX] = u^T V[X] u$
• Def. Parametric: Unknown expected value and variance, but normal distribution; Nonparametric: unknown distribution.

Tips

• Remember to center your predictors.
  • A lesson from a “Large-Scale, Multi-Lab” study. https://onlinelibrary.wiley.com/doi/10.1111/desc.70029

Data Preparation

• Use meaning codes for missing values [1]
• A ‘data dictionary’ that includes the name of each variable, a description, and other information. [1]

Resources

https://rkabacoff.github.io/datavis/

General principle

• When there are multiple choices (exclude outliers or not), instead of creating multiple data sheets, create new variables that indicate which treatment of data is used.
• Avoid HARKing: Hypothesizing After the Results are Known

Mix-effects models

Bayesian Stat tools

Hypothesis Testing

• Gigerenzer’s interpretation, from Livengood

  • Fisher

    • Science is an activity in which we make inferences in order to have well-supported beliefs about the world.
    • A preliminary way of measuring the evidence against a specified statistical hypothesis.
    • When no much prior study is available (do not know a lot about the problem), report the exact p-value. Otherwise, use more nuanced method to aggregate the evidence.
    • p-value is not inferential and has no thing to do with hypothesis rejection. Do not use the conventional 0.05.

  • Neyman

    • A way of deciding between two competing hypotheses (a null H0 and a specific alternative HA to the null).
    • Decide about $\alpha, \beta$ and sample size before the experiment, based on subjective cost-benefit considerations. These define a rejection region for each hypothesis.
    • Type I error with frequency $\alpha$: reject the null hypothesis when it is true. Falsepositive. As Neyman’s method being applied over and over, $\alpha$ converges to the chosen critical value.
    • Given H0 and HA, the sample size determines frequency $\beta$ of Type II error (false negative; accept the null when the alternative is true).