insideout Get Pro

14 · Number formats

Pro

Inference and Serving, lesson 14

Number formats

What FP32, BF16, FP8, INT8 and INT4 each keep and lose.

You know a model is billions of stored numbers, and from "Arithmetic intensity" that decode spends most of its time reading them.

Part 1. Sign, exponent, mantissa

Scientists write 6.5 as 1.625 × 2². Computers store decimals the same way, in three parts: a sign (plus or minus), a power of two, and the digits that get multiplied by it. Here is 6.5 in a 16-bit format.

Example 1.1

Reading 6.5 out of 16 bits

  1. 0100011010000000

    16 bits, split 1, 5 and 10.

  2. 0→positive

    The first bit is the sign. 0 means plus.

  3. 10001=17
    17−15=2→× 2²

    The next 5 bits give a power of two. They're stored with 15 added, so negative powers fit too.

  4. 1010000000→1.101 in binary
    1+0.5+0.125=1.625

    The last 10 bits are the digits after the point. A leading 1 is assumed and never stored.

  5. 1.625×4=6.5

    Put them together and you get the number back.

Exponent and mantissa

The exponent bits pick a power of two, so they set the range: how big or how small a number can get. The mantissa bits are the digits, so they set the precision: how finely you can tell nearby numbers apart. Floating point formats differ only in how many bits go to each.

Your turn

In that format (FP16), what's the gap between 1.0 and the next number up? The mantissa has 10 bits.

Ten bits after the binary point means steps of 2⁻¹⁰ = 1/1,024, about 0.001, near 1. Near 1,000 the gap is 0.5, because the same 10 bits of digits now sit under a bigger power of two. Floating point keeps the number of digits fixed, so the gaps grow with the number.

Part 2. Range against precision

Keep going with Pro

You've done the first part. Pro opens the rest of this lesson, and every other one.

Still to come
  1. Range against precision
  2. Integers need a scale
  3. What a model weighs
See Pro plansI have Pro: sign in
Next lessonQuantizing weightsMap weights to small integers with a scale and a zero point. Start lesson 15

Inference and Serving overviewSpotted a mistake? Tell us.