In this article, I share how I built a semantic similarity search API using FastAPI, Sentence-BERT (SBERT), and PostgreSQL with pgvector.
The idea was to test whether we can achieve context-aware text search directly inside Postgres, without adding a separate vector database.
It turned out to be a simple yet powerful setup — perfect for real-world applications and quick POCs alike.

Full Detailed Implementation: Read the complete walkthrough here →

Production notes (October 2026)

These notes add the diagnostic queries, decision criteria and validation steps the topic needs on a production system.

The simple architecture

A FastAPI endpoint receives text, Sentence-BERT produces an embedding, and PostgreSQL stores the vector alongside document metadata. A query embeds the user's text with the same model, orders candidates by the chosen distance operator and returns the nearest documents. This is a useful proof of concept because the vector data stays close to relational filters and application records.

Define the query and the quality target

Start with a fixed embedding model, its dimension, a distance metric appropriate to that model and a representative set of user queries. For example, with cosine distance and tenant-scoped documents, the database query may look like this:

SELECT id, title, embedding <=> $2::vector AS distance
FROM documents
WHERE tenant_id = $1
ORDER BY embedding <=> $2::vector
LIMIT 10;

Measure an exact-search baseline for each query cohort, then score an approximate index against it using recall at 10 and returned-result count. The target should reflect the product: a fast result set missing relevant documents is not a success. Keep the model version with the stored vector so re-embedding and rollback can be planned.

Production decisions beyond the API code

  • Embedding consistency: store the model name and version used for each vector. A model change may require re-embedding and index rebuilding; vectors from different models are not automatically comparable.
  • Distance and index: choose a distance metric that matches the model and benchmark exact search before adding approximate nearest-neighbor indexing. Compare HNSW and IVFFlat at a fixed recall target, not just peak query speed.
  • Filters: test the real tenant and metadata predicates. Approximate index scans may return too few results after filtering unless scan settings or index design are adjusted.
  • End-to-end latency: measure embedding generation, network time, SQL execution and result hydration separately. Optimizing only the database query can miss the largest cost.

Filtered search is the production trap

An HNSW or IVFFlat scan can find approximate neighbors across the index before the SQL filter removes other tenants or categories. With a selective filter, LIMIT 10 can return fewer than ten rows or lower recall even when the database is fast. A B-tree on tenant_id helps some exact paths, but it does not automatically prefilter every approximate vector scan.

Benchmark common, rare and largest-tenant filters separately. In pgvector 0.8.0 and later, iterative scans can search farther for qualifying rows, subject to work limits. Partial indexes for a small set of filter values or partitioning for suitable tenant groups are other options; each adds build, memory and maintenance cost. The pgvector filtering documentation describes these choices.

Measure the API, not only the SQL

Record p50 and p95 time for embedding generation, database execution, network transfer and result hydration under realistic concurrency. Then track recall, result count, index size, build time, insert latency and memory as the collection grows. Keep exact search as a baseline for small filtered cohorts; an approximate index is not mandatory for every query.

For each important query cohort, maintain an exact-search result set as a recall baseline. Track latency, recall, returned-result count, memory and write/index-maintenance cost as data grows. The pgvector documentation describes the index types and filtering behavior; our HNSW versus IVFFlat guide turns those details into a benchmark plan.