Concept 1 of 71 min
Regression and its traps
Slope against R²
| Correlation | 0.4 |
|---|---|
| σy | 6 |
| σx | 3 |
- Slope = 0.4 × 6 ÷ 3 = 0.8
- R² = 0.42 = 0.16
y moves 0.8 per unit of x, yet x explains only 16% of its variance.
A stronger correlation
| Correlation | 0.6 |
|---|
- R² = 0.62 = 0.36
Going from 0.4 to 0.6 more than doubles the variance explained, from 16% to 36%.
Heteroskedasticity leaves the coefficients unbiased but the usual standard errors wrong. Near-collinear regressors give unstable coefficients, and leaving out a variable that drives both sides biases the slope.
The slope carries the units of y per unit of x, so it is not the correlation. A correlation of 0.6 explains only about a third of the variance.
With heteroskedastic residuals, t-statistics mislead: use robust (White) standard errors. When 2 regressors move together, the model cannot tell their effects apart, so estimates stay unbiased but imprecise.
A strong link between a stock's returns and its advertising spend can come from the market driving both: add it as a control. Adding a variable never lowers R², so adjusted R² penalises extra regressors.
- Taking the correlation as the slope.
- Thinking heteroskedasticity biases the coefficients.
- Judging a model with more regressors on plain R².
You find that a stock's returns rise with something it reports each quarter. What would make you doubt the relationship?
A complete answer covers
- An omitted variable, such as the market, driving both.
- Non-stationary series producing spurious correlation.
- Standard errors that ignore heteroskedasticity or autocorrelation.
- Data mining: many variables tried, one found.
Also in our technical interviews, on camera, with a score.
Concept 2 of 71 min
Correlation, variance of sums and spurious links
The why, the charts, the worked example and the interview questions are in Essentials, with every quiz.
See the plansSign inConcept 3 of 71 min
Significance, errors and intervals
The why, the charts, the worked example and the interview questions are in Essentials, with every quiz.
See the plansSign inConcept 4 of 71 min
Normal quantiles, skewness and fat tails
The why, the charts, the worked example and the interview questions are in Essentials, with every quiz.
See the plansSign inConcept 5 of 71 min
Return arithmetic and robust statistics
The why, the charts, the worked example and the interview questions are in Essentials, with every quiz.
See the plansSign inConcept 6 of 71 min
Fitting models: overfitting, regularisation, metrics
The why, the charts, the worked example and the interview questions are in Essentials, with every quiz.
See the plansSign inConcept 7 of 71 min
AdvancedThe Sharpe ratio luck alone would find
The why, the charts, the worked example and the interview questions are in Essentials, with every quiz.
See the plansSign inRecap
- OLS slope = correlation × σy ÷ σx; with one regressor, R² is the squared correlation.
- Var(X + Y) = Var(X) + Var(Y) + 2 × ρ × σX × σY; correlate returns, not price levels.
- A p-value is the chance of data this extreme if the null is true, not the chance the null is true; test many strategies and some will pass by luck.
- Standard error = σ ÷ √n; a 95% interval is ± 1.96 standard errors; 2.33 is the one-sided 1% quantile.
- Sharpe ratios scale with √time; log returns add; the median resists outliers; a big gap between training and test fit means overfitting.
25 questions on Statistics and machine learning
Timed, like the real one, drawn from a bank of 33. Every answer explained.