The instinct
Every retrieval feature starts with the same conversation. Someone says the word embeddings, someone else names a vector database, and within an hour there is a new piece of infrastructure on the diagram that nobody has operated before.
We did that on our first internal product. Then we asked what it was buying us at our actual size, which was tens of thousands of documents and a handful of concurrent users, not tens of millions and thousands.
What we measured
We built the same retrieval path twice: once on a dedicated vector store, once on Postgres with pgvector and an ordinary index. Same corpus, same embedding model, same top-k, same reranking step afterwards.
At 100,000 chunks the difference in median latency was small enough that the network hop to a separate service ate most of it. Recall was indistinguishable once reranking was in place. The dedicated store won clearly on paper only when we simulated a corpus roughly twenty times larger than anything we had.
What it cost to keep Postgres
One database to back up, one connection pool, one set of credentials, one migration story. Retrieval sat in the same transaction as the metadata it filtered on, which removed an entire class of consistency bug we did not have to write code to prevent.
The real cost was discipline: keeping the embedding column narrow, keeping the index maintenance in the same migration path as everything else, and writing down the number at which we would revisit the decision.
When we would switch
We wrote the trigger into the architecture note: sustained p95 over 400ms on retrieval alone, or a corpus past roughly a million chunks, or a filtering pattern the index cannot serve. None of those are close yet.
The point is not that dedicated vector databases are wrong. It is that a second datastore is a permanent operational cost, and it should be paid when a measurement asks for it rather than when a diagram suggests it.