insideout Get Pro

06 · Indexes at scale with HNSW

Pro

Search and Retrieval, lesson 6

Indexes at scale with HNSW

Approximate nearest neighbours, fast enough for millions of vectors.

"Vector search": nearest-neighbour search, and the bucket trick for skipping most vectors. This lesson is about what changes when there are millions of them.

Part 1. What a million vectors cost

Each number in an embedding is usually stored as a 32-bit float: 4 bytes. A 768-number embedding is 3,072 bytes. That sounds small until you multiply.

Your turn

You have 10 million chunks with 1,536-number embeddings, stored as 4-byte floats. About how much memory do the vectors need?

10,000,000 × 1,536 × 4 bytes = 61.44 billion bytes, about 61 GB, before any index structure or the chunk text. That's why vector size and storage format matter at scale.

Speed scales the same way. Checking every vector means 10 million × 1,536 multiply-adds, about 15 billion, for each question. For 10,000 chunks, though, it's about 15 million, which a laptop does in a blink. Below some size, exact search is the simplest right answer.

Exact search

Comparing the query with every stored vector (also called brute force or flat search). Always finds the true nearest neighbours. The baseline every faster index is measured against.

Part 2. A graph you can walk

Keep going with Pro

You've done the first part. Pro opens the rest of this lesson, and every other one.

Still to come
  1. A graph you can walk
  2. Shrinking the vectors
  3. What a vector database adds
See Pro plansI have Pro: sign in
Next lessonFilters and metadataCombine vector search with dates, owners and permissions. Start lesson 7

Search and Retrieval overviewSpotted a mistake? Tell us.