Skip to Content

Vector Embeddings: From Text to Numbers

When to use

Convert text, images or audio into numeric vectors such that semantically similar items end up close together in the vector space.

Analogy

Translating words into points on a giant map. ‘King’ and ‘queen’ end up near each other; ‘king’ and ‘banana’ end up far apart. The map is the model.

Data-flow diagram

  text           -->  [0.12, -0.04, ..., 0.83]   (1536-dim)
  'king'                  ^
                          |  cosine similarity 0.86
                          v
  'queen'         -->  [0.10, -0.05, ..., 0.81]   (1536-dim)

Deep explanation

An embedding model turns any input into a fixed-size vector of floats (typical sizes: 384, 768, 1024, 1536, 3072). Cosine similarity between two vectors tells you how semantically close the inputs are. Modern embedding models: text-embedding-3-small (1536, $0.02 per M), text-embedding-3-large (3072, $0.13 per M), voyage-large-2 (1024), cohere-embed-v3 (1024), and open-source all-MiniLM-L6-v2 (384). Pick the model that matches your domain’s vocabulary and your budget.

Examples

Example 1

from openai import OpenAI
oai = OpenAI()
resp = oai.embeddings.create(model='text-embedding-3-small', input='the cat sat on the mat')
vec = resp.data[0].embedding
print('dim:', len(vec), 'first 5:', vec[:5])

OpenAI embeddings give high quality at low cost per 1M tokens; pick the small model first.

Example 2

# open-source option
from sentence_transformers import SentenceTransformer
m = SentenceTransformer('all-MiniLM-L6-v2')   # 384-dim
v = m.encode('the cat sat on the mat')
print('dim:', v.shape, 'norm:', float((v*v).sum())**0.5)

sentence-transformers runs locally — zero API cost, useful for prototyping or air-gapped setups.

Example 3

# cosine similarity
import numpy as np
def cos(a, b):
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))
v1 = m.encode('how to refund an order')
v2 = m.encode('how to return a product')
print('cosine:', cos(v1, v2))   # should be high

Cosine similarity ignores vector magnitude; correct when embeddings are normalised or direction matters.

Common mistake

Swapping embedding models mid-project without re-indexing. Different models live in different spaces; old indexes are unwritable with new ones.

Key takeaway

Embeddings = vectors from a model. Cosine similarity = how close. Pick a model and stick with it; re-embed the whole corpus if you switch.

Production Failure Playbook

Failure scenario 1: model-swap-without-reindex

Failure scenario 2: wrong-dim-column