All posts

Deriving the Central Limit Theorem

Central Limit Theorem
Characteristic functions
Taylor series
Showing why the sample mean of independent and identically distributed random variables converges to a normal distribution as the sample size tends to infinity.
Nov 7, 2024
Chris Kelly

Deriving the Weak Law of Large Numbers

Law of Large Numbers
The weak law of large numbers states that the sample mean \(\bar{X}_n\) converges to the population mean \(\mu\) as the sample size \(n \rightarrow \infty\).
Jun 14, 2024
Chris Kelly

Generalized Least Squares

Linear Models
Generalized Least Squares
Applying a weight matrix to correct for non-homoskedastic error variance can be more efficient than OLS with sandwich errors, aka GLS.
Jun 11, 2024
Chris Kelly

Generalized Linear Models from scratch

Generalized Linear Models
Maximum Likelihood
Python
Deriving (and implementing in python) a generic way to optimize GLMs, by using the exponential family form for Poisson, Bernoulli and Gaussian distributions.
May 28, 2024
Chris Kelly

The gaussian integral

Gaussian
Fundamentals
Why does the area under the gaussian integral \(\int_{-\infty}^{+\infty}{e^{-x^2}\,dx}=\sqrt{\pi}\)?
May 13, 2024
Chris Kelly

Deriving the normal distribution

Gaussian
Fundamentals
Where does the gaussian distribution come form? We derive if from first principles i.e. proving the Herschel-Maxwell theorem.
May 11, 2024
Chris Kelly

Deriving the poisson distribution from binomial

Binomial
Poisson
How extending the binomial distribution to an infinite number of trials derives the poisson distribution.
Feb 28, 2024
Chris Kelly

Estimating population variance from a sample

Fundamentals
Bias
Why is sample variance divided by \(n-1\), but population variance by \(N\).
Feb 26, 2024
Chris Kelly

Logistic Regression as a latent variable model

Maximum Likelihood
Generalized Linear Models
Taking a “latent variable” approach to modelling bernoulli probabilities using maximum likelihood estimation and the logit link function.
Feb 25, 2024
Chris Kelly

Shrinkage priors for Lasso and Ridge

Bayes
Maximum Likelihood
Regularization
Laplace
Showing Laplace and Gaussian priors in Bayesian regression are equivalent to Lasso and Ridge regularization
Feb 24, 2024
Chris Kelly

OLS vs MLE with gaussian noise

Linear Models
Maximum Likelihood
Gauss-Markov
Why MLE finds the same coefficients as OLS when assuming model errors are generated from a mean-zero gaussian probabilistic process
Feb 23, 2024
Chris Kelly

Sandwiches: robust covariance error estimation

Linear Models
Gauss-Markov
Standard errors
Estimating the correct coefficient variance when relaxing homoskedastic error assumptions
Feb 22, 2024
Chris Kelly

BLUE coefficients: bias and efficiency

Linear Models
OLS
Gauss-Markov
Python
Proving that the OLS coefficient is the best linear unbiased estimator (under the Gauss-Markov assumptions).
Feb 21, 2024
Chris Kelly

Deriving OLS coefficients (multivariate)

Linear Models
OLS
Python
OJS
Deriving a unique, analytical solution to the set of coefficients that minimize the sum of squared residuals.
Feb 20, 2024
Chris Kelly

Mean Squared Error vs Mean Absolute Error

Cost functions
Minimizing MAE tends predictions towards the sample median, whereas for MSE predictions tend towards the sample mean.
Feb 8, 2024
Chris Kelly

Differentiation formulae

Fundamentals
Deriving why \(\frac{d}{dx} \left(x^{n} \right) = nx^{n-1}\)
Feb 8, 2024
Chris Kelly
No matching items