Why split documents into chunks before embedding, and what splits lose precision?

Category: Database & RAG

Answer

A 50-page PDF embedded as one vector is a soup — the retrieval finds the document but loses fine-grained locality. Chunks split the document into semantically coherent slices (sentence-based, recursive by paragraph, code-aware by delimiter). Each chunk becomes one row with its own vector; retrieval returns K focused rows instead of one fuzzy blob.

Concrete examples from the fca project context

Example 1

Fixed window 512 tokens, overlap 64 — default for prose.

Example 2

RecursiveCharacterTextSplitter respects paragraph + sentence boundaries first.

Example 3

Code-aware splitter keeps def/class bodies intact — different chunk_size, different language.

Data flow / flow chart

PDF -> page -> recursive splitter -> chunk (512 tok, overlap 64)
  embed(chunk) -> chunk.embedding -> chunks table
  retrieve top-K -> context

Takeaway

Chunk shape decides retrieval quality more than the model does. Re-tune chunk size before re-embedding.