Start with an exact baseline

Approximate nearest-neighbor indexes trade some recall for speed. Before comparing HNSW and IVFFlat, run representative queries with exact search and save the top-k results. That gives you a ground truth for recall at k, not merely a lower latency number. Include common and difficult queries, multiple tenants, metadata filters and the vector count you expect after growth.

What changes between HNSW and IVFFlat?

HNSW uses a graph and can be built before data is loaded. It often offers a strong speed–recall trade-off, with more memory and slower build or maintenance cost. IVFFlat groups vectors into lists and is best created after representative data is available; its probe count controls how many lists are searched. Neither is automatically the right choice for a small collection, a rapidly changing dataset or a heavily filtered query.

Filtering is a common surprise. With approximate indexes, a metadata predicate can reduce the candidates left after the vector scan, so a query requesting ten results may return fewer than ten. The answer may involve filter design, a different index strategy, higher search effort or iterative scans, depending on pgvector version and workload. Test the returned-result count as well as latency.

A benchmark that supports a decision

  1. Fix the embedding model, distance metric, top-k value and dataset for all runs.
  2. Record exact-search results, then compare HNSW and IVFFlat at several search-effort settings. For HNSW, include hnsw.ef_search; for IVFFlat, include ivfflat.probes.
  3. Measure recall@k, p50/p95 latency, returned-result count and the complete SQL query with filters and joins.
  4. Measure index size, build time, insert/update cost and memory under realistic concurrency.
  5. Repeat after a realistic data-growth and deletion cycle. Do not choose from a single warm-cache microbenchmark.

The pgvector project documentation describes index types, search settings and iterative scans. For the API architecture that feeds this workload, see the semantic search API guide. For a production review, see pgvector performance capability.