Pick how you split documents into chunks before embedding. Chunk shape drives retrieval quality more than the embedding model does.
Cutting a long loaf of bread into slices. Too thin, you lose context. Too thick, you bury the answer. The cut pattern also matters: equal slices (fixed), overlapping slices (sliding), or by structure (by section / by code block).
document chunk-1 chunk-2 chunk-3
xxxxxxxxxxxxxxxxxxxxx .......... .......... ..........
| |
overlap=N/4
strategies: fixed window | sliding window | recursive by delimiter | semantic by similarity
Fixed window: split every N tokens. Simple, predictable; loses context at chunk boundaries. Sliding window: overlap M tokens between consecutive chunks; preserves context across boundaries; doubles embedding cost. Recursive: split by paragraph/section/code-block boundaries first, then by token cap. Semantic: split at points where semantic similarity between consecutive sentences drops below a threshold — usually requires an extra embedding pass per chunk. Default to fixed or sliding; switch to recursive when structure matters (markdown sections, code); reach for semantic when domain-specific. Always also extract overlap into a metadata field.
# fixed window
from langchain.text_splitter import CharacterTextSplitter
sp = CharacterTextSplitter(chunk_size=512, chunk_overlap=64)
chunks = sp.split_text(long_doc)
print('fixed:', len(chunks), 'chunks; first 60 chars:', chunks[0][:60])
Fixed window is fast and predictable; default for generic prose.
# recursive (by delimiter)
from langchain.text_splitter import RecursiveCharacterTextSplitter
sp = RecursiveCharacterTextSplitter(chunk_size=512, chunk_overlap=64,
separators=['\n\n', '\n', '.', ' '])
chunks = sp.split_text(long_doc)
Recursive respects paragraph and code-block structure — better for technical documents.
# code-aware
from langchain.text_splitter import RecursiveCharacterTextSplitter, Language
sp = RecursiveCharacterTextSplitter.from_language(Language.PYTHON,
chunk_size=512, chunk_overlap=64)
chunks = sp.split_text(python_source)
Code-aware splitter keeps def/class boundaries intact — best retrieval for source code.
Embedding whole documents as one vector. Splits are the foundation of retrieval; never skip chunking.
Default to fixed window. Upgrade to recursive by delimiter for technical docs. Use code-aware splitter for source code. Tune overlap to 10-20 percent of chunk size.