insideout Get Pro

08 · Estimating covariance

Pro

Portfolio and Risk, lesson 8

Estimating covariance

Noisy estimates and why shrinkage helps.

The covariance matrix from Diversification, the minimum-variance portfolio and shrinkage from The efficient frontier, and overfitting from Statistics and Regression.

Part 1. Thirty stocks, sixty days

Thirty simulated stocks in three sectors. Estimate their covariance from 60 days, build the lowest-risk portfolio, then see how it really swings over the next 250 days.

Shrink the estimateLive

With no shrinkage, the optimiser trusted every noisy correlation and promised a calm portfolio that turned out rougher. Blending the estimate with a plain target, where every pair has the same correlation, gave up a little detail and removed a lot of noise. With more data the formula leaned back towards the sample.

Shrinkage target and intensity

The plain matrix you blend towards is the shrinkage target: here, the sample's own variances with every correlation set to their average. The target's share of the blend, δ ("delta") from 0 to 1, is the shrinkage intensity.

Your turn

A formula picks a shrinkage intensity of 0.77 from 60 days of returns. You now have 1,000 days of the same stocks. What do you expect it to pick?

Shrinkage pays when the sample is noisy. More data means less noise, so the formula trusts the sample more. The code below finds 0.11 at 1,000 days.

Part 2. Too many numbers

Keep going with Pro

You've done the first part. Pro opens the rest of this lesson, and every other one.

Still to come
  1. Too many numbers
  2. Why the forecast flatters
  3. A worked problem
  4. The same thing, for real
See Pro plansI have Pro: sign in
Next lessonSizing betsKelly, fractional Kelly, and the cost of betting too much. Start lesson 9

Portfolio and Risk overviewSpotted a mistake? Tell us.