Skip to Content

Chroma (Lightweight Local Vector Store)

When to use

Prototype a RAG pipeline quickly with a self-hosted, file-backed vector store. Defer managed infrastructure until you need scale.

Analogy

A spiral notebook for embeddings — runs locally, ships with the app, persists to disk. Perfect for notebooks and prototypes.

Data-flow diagram

  import chromadb
  client = chromadb.PersistentClient(path='./chroma')
  col    = client.get_or_create_collection('docs')
  col.add(ids=['a'], documents=['...'], embeddings=[...])
  col.query(query_embeddings=[...], n_results=3, where={'tenant':'acme'})

Deep explanation

Chroma is the lightest production-ready vector store. PersistentClient writes to disk; the ephemeral in-memory client is the simplest possible dev loop. Add documents with optional pre-computed embeddings OR pass through a built-in or callable embedder. Use Chroma when fog/proto matters more than billion-scale performance.

Examples

Example 1

import chromadb
client = chromadb.PersistentClient(path='./chroma')
col    = client.get_or_create_collection('docs', metadata={'hnsw:space':'cosine'})
col.add(ids=['a','b','c'], documents=['alpha text','beta text','gamma text'],
        embeddings=[v_for_a, v_for_b, v_for_c])

PersistentClient writes to disk — restart safe, no separate service.

Example 2

# query with metadata filter
results = col.query(query_embeddings=[q_vec], n_results=3,
                    where={'tenant':'acme'}, include=['documents','metadatas'])
print(results)

hnsw:space sets cosine vs euclidean distance metric.

Example 3

# with built-in embedding function (uses all-MiniLM-L6-v2 by default)
from chromadb.utils import embedding_functions
col = client.get_or_create_collection('docs',
    embedding_function=embedding_functions.DefaultEmbeddingFunction())
col.add(ids=['a','b','c'], documents=['alpha','beta','gamma'])

Built-in embedding function keeps prototype code minimal.

Common mistake

Using chromadb in production at scale. Use it for prototyping as designed; switch to pgvector or Pinecone at scale.

Key takeaway

Chroma = local-first, persistent, easy RAG prototype. Lift and shift to pgvector or Pinecone when scale or sharing demands it.

Production Failure Playbook

Failure scenario 1: metadata-filter-syntax

Failure scenario 2: memory-blowup-on-embed