insideout Get Pro

01 · Samples and estimates

Free

Statistics and Regression, lesson 1

Samples and estimates

A sample mean, its standard error, and why more data helps slowly.

You never see a stock's true average return, only a sample of its days. This lesson measures how far a sample's average can miss the truth, and how much data it takes to miss by less. You need averages, standard deviations and square roots.

By the end, you can:

  1. Compute a sample's average and standard deviation, and read them as guesses.
  2. Work out the standard error of an average from the spread of one day.
  3. Work out how much more data it takes to shrink a standard error.
  4. Work out how many days it takes to tell a small average from zero.

Part 1. A sample average is a guess

Compute a sample's average and standard deviation, and read them as guesses.

GULL is a practice stock. Its daily returns come from a fixed process with a true average, but nobody tells you what it is. All you see is some of its days, a sample, and from them you compute numbers that stand in for the hidden ones.

Example 1.1

Five days of GULL

GULL returns +3.5%, −1.5%, +1.5%, −0.5% and −0.5% on five days. What are the sample's average and standard deviation?

  1. Average the days(3.5 − 1.5 + 1.5 − 0.5 − 0.5) / 5 = 0.5%

    This is the sample average, written x̄ and read "x bar".

  2. Measure each day from the average3.0, −2.0, 1.0, −1.0, −1.0

    Each return minus 0.5%. These deviations always add up to zero.

  3. Square them and add9 + 4 + 1 + 1 + 1 = 16

    Squaring stops the ups and downs from cancelling.

  4. Divide by n − 1, then take the root√(16 / 4) = 2.0%

    Five days, so divide by 4. Why 4 and not 5 comes below.

AnswerAn average of 0.5% a day, and a standard deviation of 2.0%.

GULL is simulated, so we can peek: its true average is 0.1% a day, and its true standard deviation is 2%. The sample's spread came out right, but its average landed 0.4 points high. Nothing went wrong. These five days just happened to include a good one.

−4%−2%0%+2%+4%One day20 single days: sd 1.98%Average of 10 days20 averages: sd 0.62%true average 0.1%
Figure 1Twenty single days of GULL, and twenty averages of 10 days each. Every average misses the true 0.1%, but they bunch far closer to it than single days do.

Estimate

A number computed from a sample to stand in for a hidden one is an estimate. The sample average x̄ estimates the true average μ ("mu"), and the sample standard deviation s estimates the true one, σ ("sigma"). Σ means "add up":

s = √( Σ(x − x̄)² / (n − 1) )

Why n − 1? Each day is measured from the sample's own average, and that average sits closer to its own days than the true average does. So the squares come out a little small, and dividing by n − 1 instead of n makes up for it. With five days that matters. With five hundred, it hardly does.

Your turn

A stock returns +2.2%, −1.8%, +2.2%, −1.8% and +0.2% on five days. What are its sample average and sample standard deviation?

The returns add to 1.0, so x̄ = 0.2%. The deviations are 2, −2, 2, −2 and 0, their squares add to 16, and √(16 / 4) = 2.0%. Dividing by 5 gives 1.79%. Forgetting the square root leaves 4.0, the variance. 1.0% is the sum, not the average.

Part 2. How far an average wobbles

Work out the standard error of an average from the spread of one day.

The averages in Figure 1 scatter by a fairly steady amount. You can predict that amount before taking any sample, from two numbers: how much one day moves, and how many days you average.

Example 2.1

The wobble of a 10-day average

GULL's daily returns have a standard deviation of 2%, and its days are independent. How far does an average of 10 days typically land from the true average?

  1. Variance of one dayσ² = 2² = 4

    Variance is the standard deviation squared.

  2. Variance of the sum of 10 days10 × 4 = 40

    For independent days, variances add.

  3. Variance of the average40 / 10² = 0.4

    The average is the sum divided by 10, and dividing a number by 10 divides its variance by 100.

  4. Back to a standard deviation√0.4 = 0.63%

    The square root returns to percent.

AnswerAbout 0.63%. The twenty averages in Figure 1 had a standard deviation of 0.62%.

Why so much less than 2%? Inside a sample, good days and bad days partly cancel. The sum of 10 days spreads only √10 times as wide as one day, not 10 times as wide, and the average divides that by 10. That leaves σ / √10.

Standard error

How far an estimate typically lands from the truth, measured as a standard deviation, is its standard error, SE for short. For an average of n independent days, each with spread σ:

SE = σ / √n = 2% / √10 = 0.63%

You rarely know σ, so you put the sample's own s in its place. For the five days in part 1, SE ≈ 2.0% / √5 = 0.89%. Their average of 0.5% is give or take about 0.9%, so those five days can't even say whether GULL's average is above zero.

Two traps. The standard error describes the average, not single days: GULL's days still move about 2% each. And the rule needs independent days. Days that move together cancel less, so their true standard error is bigger than σ / √n.

Your turn

A stock's daily returns have a standard deviation of 1.5%. What's the standard error of the average of 25 days?

SE = 1.5% / √25 = 1.5% / 5 = 0.3%. 0.06% divides by 25 instead of √25. 7.5% is the spread of the sum of 25 days, 1.5% × 5, before dividing by 25.

Part 3. Why more data helps slowly

Work out how much more data it takes to shrink a standard error.

The √n in the standard error has a cost. To halve the error of an average, doubling the data isn't enough.

Example 3.1

Halving the error

Ten days of GULL give an average with a standard error of 0.63%. How many days bring it to 0.32%? And to 0.063%, a tenth?

  1. Halve the error√n × 2 → n × 4 = 40 days

    The error is 2% / √n, so it halves when √n doubles, and √n doubles when n is four times as big.

  2. Cut it to a tenth√n × 10 → n × 100 = 1,000 days

    Ten times smaller needs √n ten times bigger, so a hundred times the days.

  3. Put it in years1,000 / 252 ≈ 4 years

    A year has about 252 trading days.

Answer40 days for half the error, and 1,000 days, about four years, for a tenth.

Each new day is worth less than the last. Going from 10 to 40 days cut the error by 0.32 points. Going from 40 to 160, four times the data again, cut it by only 0.16.

0.2%0.4%0.6%0.8%0160320480640days averaged10 days: 0.63%40 days: 0.32%160 days: 0.16%640 days: 0.08%2% / √neach 4 × the days halves it
Figure 2The standard error of GULL's average, 2% / √n. Each fourfold increase in days halves it: 0.63% at 10 days, 0.32% at 40, 0.16% at 160 and 0.08% at 640.

The square-root rule

A standard error falls with the square root of the sample size. To make it k times smaller, you need k² times the data.

4 × the data → SE ÷ 2; 100 × the data → SE ÷ 10

The rule assumes the extra data comes from the same process. For a stock, more days means older days, from markets that may have worked differently. Data long enough to be precise can be too old to be relevant.

Your turn

Your estimate of a mean has a standard error of 0.3%. You need 0.03%. How much more data does that take?

The error falls with √n. To make it 10 times smaller, √n has to grow 10 times, so n grows 100 times.

Part 4. Small edges need years of data

Work out how many days it takes to tell a small average from zero.

Here is the standard error at work, in a question in the style of quant research interviews. A strategy earns a little on average and moves a lot from day to day. How long before its track record shows the edge?

Example 4.1

A strategy with a small edge

A strategy's daily returns have a standard deviation of 1%, and its true average is 0.05% a day. How many days before the standard error is smaller than the average? And before the average sits two standard errors above zero?

  1. Write the standard errorSE = 1% / √n

    σ = 1%, over n days.

  2. Make it smaller than the average1% / √n < 0.05% → √n > 20 → n > 400

    1 / 0.05 = 20, and squaring gives 400.

  3. Ask for two standard errors0.05% / (1% / √n) > 2 → √n > 40 → n > 1,600

    Twice the distance needs √n twice as big, so four times the days.

  4. Put it in years1,600 / 252 ≈ 6.3 years

    About 252 trading days in a year.

AnswerMore than 400 days for the first, and more than 1,600 days, over six years, for the second.

Why two standard errors? A strategy with a true average of zero rarely shows a sample average two standard errors above it, as the next two lessons show. So two standard errors is the usual bar for "probably not luck".

The working fits on one line. To sit k standard errors above zero, you need n > (k × σ / μ)² days. Halve the edge and you need four times as long. And six years assumes the strategy and its market stay the same the whole time.

Your turn

A strategy averages 0.1% a day with a daily standard deviation of 1.5%. About how many days before its average sits two standard errors above zero?

You need 0.1% / (1.5% / √n) > 2, so √n > 2 × 1.5 / 0.1 = 30 and n > 900, about three and a half years. 30 forgets to square. 225 asks for only one standard error, and 450 doubles that. Twice the standard errors needs four times the days.

Where you'll meet it

In interviews

Standard error questions are common in quant research and data science interviews: how the error of a mean shrinks with n, how much data halves it, and why the sample standard deviation divides by n − 1. The part 4 question often comes turned around: "how long before you'd know this strategy works?" Start from the edge over the spread, then square.

On the job

Researchers distrust an average return measured over a few years, because its standard error is often as big as the average. Volatility is far easier to pin down. For normally distributed returns, the standard error of a sample standard deviation is about σ / √(2n), so one year of daily data pins volatility to within about 4.5% of itself, while the same year leaves the average in doubt. Fat tails widen that error, but the gap stays large.

The square-root rule, in Python
import random
import statistics

days = [3.5, -1.5, 1.5, -0.5, -0.5]  # five days of GULL, in %
print(statistics.fmean(days), statistics.stdev(days))  # stdev divides by n - 1

random.seed(1)
mu, sigma = 0.10, 2.0  # GULL's daily return in %: the truth we pretend not to know

def sample_mean(n):
    return statistics.fmean(random.gauss(mu, sigma) for _ in range(n))

for n in [10, 40, 160, 640]:
    means = [sample_mean(n) for _ in range(2000)]
    spread = statistics.stdev(means)
    print(f"n = {n:>3}: spread of the averages {spread:.3f}%, sigma / sqrt(n) = {sigma / n ** 0.5:.3f}%")

# Output:
# 0.5 2.0
# n =  10: spread of the averages 0.639%, sigma / sqrt(n) = 0.632%
# n =  40: spread of the averages 0.323%, sigma / sqrt(n) = 0.316%
# n = 160: spread of the averages 0.160%, sigma / sqrt(n) = 0.158%
# n = 640: spread of the averages 0.078%, sigma / sqrt(n) = 0.079%

Each line draws 2,000 samples of n days and measures how much their averages spread. The simulation and the formula agree to within 0.01 percentage points every time.

Check yourself

Four questions from across the lesson. Try each one before you look back.

1

You average 10 days of GULL and get +0.6% a day. Someone else averages 10 different days and gets −0.3%. What does that tell you?

With a daily spread of 2%, averages of 10 days wobble by about 0.6% either way, so two estimates 0.9% apart are ordinary. Averaging the two helps a little, but that's still only a 20-day estimate.

2

GULL's daily returns have a standard deviation of 2%. You average 100 of its days. Which is right?

Averaging doesn't calm the days. It's the average that wobbles less: 2% / √100 = 0.2%. 0.02% divides by 100 instead of √100.

3

Why does the sample standard deviation divide by n − 1 rather than n?

The sample's average is fitted to its own days, so squared distances from it come out a little smaller than distances from the truth would. Dividing by n − 1 corrects that on average. It matters in small samples: with five days, it's the difference between 1.79% and 2.0%.

4

A strategy needs about 1,600 days before its average sits two standard errors above zero. A second strategy has the same daily spread and half the average return. About how long does it need?

The days needed grow with (σ / μ)². Halving μ doubles σ / μ, and squaring makes that four times the days: 6,400, about 25 years. 3,200 forgets to square.

What to remember

  1. 1A sample average is a guess

    A sample's average and standard deviation are estimates of hidden numbers: +3.5, −1.5, +1.5, −0.5 and −0.5 give x̄ = 0.5% and s = 2.0%, dividing by n − 1.

  2. 2How far an average wobbles

    The standard error of an average is σ / √n: ten days of a stock that moves 2% a day give 2% / √10 = 0.63%.

  3. 3Why more data helps slowly

    Data helps slowly: four times the days halves the standard error, and a tenth of the error takes a hundred times the days.

  4. 4Small edges need years of data

    To sit two standard errors above zero, an edge of 0.05% a day on a 1% spread needs more than (2 × 1 / 0.05)² = 1,600 days, over six years.

Next lessonConfidence intervalsNext, the standard error becomes a range: a confidence interval, the estimate give or take about two standard errors. Start lesson 2

Statistics and Regression overviewSpotted a mistake? Tell us.