Vector Database for AI Agent Memory: Qdrant vs Chroma vs pgvector

Vector Database for AI Agent Memory: Qdrant vs Chroma vs pgvector

0

Last month I rebuilt an agent’s memory layer twice in one week. The first version used Chroma, because the tutorial used Chroma, and it worked beautifully. Then it didn’t, the moment two agent instances hit the same memory store from different processes. The second version used Qdrant, and it took an hour to realize I’d made one collection per user like some kind of maniac, exactly what Qdrant’s own docs tell you not to do.

If you’ve gotten as far as giving your agent long-term memory, you’ve probably hit the question this whole piece is about: which vector database for AI agent memory actually survives contact with a second user, a second agent, or a thousand more rows than your demo had. There are three realistic candidates for where those embeddings live, and the tutorials rarely tell you which one holds up. Picking the wrong one doesn’t fail loudly. It just gets slower, or starts leaking one tenant’s memories into another’s filter, three weeks after you shipped it.

Here’s an honest vector database for AI agent memory comparison between Qdrant, Chroma, and pgvector: what each one actually does when you give it a filtered query, where the index build genuinely hurts, and the exact error message you’ll see when you get the embedding dimension wrong (because you will, at least once).

What Picking a Vector Database for AI Agent Memory Actually Involves

A RAG demo over a static document set and a live agent’s memory store are not the same workload, even though both are “vector search.” A RAG index is written once and queried a lot. Agent memory is written constantly, by multiple agents or multiple users, and almost every read needs a filter attached: “only this user’s memories,” “only this agent’s episodic log,” “only memories from this session.” Get that filter wrong, or skip it, and you’re not just risking a privacy leak between tenants, you’re also dumping irrelevant memories straight into the context window on every single call.

The index choice matters less here than one question: how does this database isolate one tenant’s vectors from everyone else’s, and what does that cost you? That’s the real question.

That’s the lens for the rest of this piece. I’m not benchmarking raw query latency on a billion vectors, nobody running an agent memory store at that scale is picking a database from a blog post. I’m looking at what happens with the second thing most people actually hit: a few hundred thousand memories, split across hundreds or thousands of agents and users, queried with a metadata filter on almost every call.

If you’ve already wired up a framework’s own memory system, like CrewAI’s unified memory, you’ve seen one side of this: a single store, a single filter key, one framework deciding how to call it. This article is one layer down, comparing the three stores themselves rather than the framework wrapper around them.

A metadata filter splitting an AI agent memory query into separate per-agent partitions.
Every real agent memory query needs a tenant filter attached, not just a raw similarity search.

pgvector: The Boring Default If You’re Already on Postgres

pgvector turns Postgres into a vector store by adding a vector column type and two index types (HNSW and IVFFlat) you build like any other index. The appeal for agent memory is specific: if your agent already uses Postgres to resume after a crash via checkpoints, or to log user records and tool calls, your memories live in the same transaction as everything else. No second system to keep in sync, no separate backup story.

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE agent_memory (
    id bigserial PRIMARY KEY,
    agent_id text NOT NULL,
    content text NOT NULL,
    embedding vector(1536)
);

CREATE INDEX ON agent_memory USING hnsw (embedding vector_cosine_ops);

-- every real agent-memory query is filtered, so filter + vector search together:
SELECT id, content, embedding <=> '[0.01, 0.02, ...]' AS distance
FROM agent_memory
WHERE agent_id = 'agent-42'
ORDER BY embedding <=> '[0.01, 0.02, ...]'
LIMIT 5;

I ran this against a real Postgres 16 instance with pgvector installed, inserted a handful of rows for two different agent IDs, and the filtered WHERE agent_id = ... query only ever returned that agent’s memories, as you’d hope. The default HNSW build parameters, per pgvector’s own documentation, are m = 16 and ef_construction = 64; you can raise both for better recall at the cost of a slower, bigger index, with CREATE INDEX ... WITH (m = 16, ef_construction = 64).

Here’s the part the “just add an index” tutorials skip: HNSW builds want to fit in maintenance_work_mem, and when they don’t, Postgres tells you so directly. I deliberately set maintenance_work_mem to 1MB, way too small, and built an HNSW index on 20,000 rows of 128-dimension vectors. Postgres didn’t error. It just raised this notice before the build:

NOTICE: Building will take significantly more time.

Not an error. Just a warning. It’s one you’ll only see if you’re watching for Postgres notices, which most ORMs swallow by default. Bump maintenance_work_mem to something like 8GB for a real build (the official docs use that exact example), and if you’re building in Docker with multiple parallel workers, raise --shm-size to match or the build can fail outright on shared memory, a gotcha that’s easy to hit the first time you containerize this.

Get the vector’s dimension wrong and pgvector won’t guess what you meant:

# inserting a 5-dimension vector into a column declared vector(8)
# psycopg.errors.DataException: expected 8 dimensions, not 5

That error came straight out of a live insert in my test database, not from documentation. It’s a hard failure at write time, which I actually prefer to a silent truncation or padding; you find the mismatch the first time you run the code, not three weeks into production.

A pgvector database connected to an HNSW index, with a low memory gauge and a slow build-time clock.
A small maintenance_work_mem setting doesn't fail the HNSW build, it just makes it much slower.

Qdrant: Built Around Many Tenants in One Collection

Qdrant is a purpose-built vector database, and it shows the moment you look at how it wants you to handle multiple agents or users. Its own multi-tenancy documentation is blunt about the instinct most people start with: “creating a separate collection for each tenant is rarely the most efficient approach.” Each collection carries its own resource overhead, and Qdrant Cloud caps a cluster at 1,000 collections by default, so the one-collection-per-user design I shipped in my first draft was never going to hold up.

The recommended pattern is one shared collection with a tenant field on every point, plus a payload index on that field marked is_tenant: true (available since Qdrant 1.11.0), which tells Qdrant to co-locate each tenant’s vectors on disk so filtered reads stay sequential instead of scattered:

from qdrant_client import QdrantClient
from qdrant_client.models import (
    VectorParams, Distance, PointStruct,
    Filter, FieldCondition, MatchValue,
    PayloadSchemaType, KeywordIndexParams,
)

client = QdrantClient(url="http://localhost:6333")

client.create_collection(
    collection_name="agent_memory",
    vectors_config=VectorParams(size=1536, distance=Distance.COSINE),
)
client.create_payload_index(
    collection_name="agent_memory",
    field_name="agent_id",
    field_schema=KeywordIndexParams(type=PayloadSchemaType.KEYWORD, is_tenant=True),
)

client.upsert(collection_name="agent_memory", points=[
    PointStruct(id=1, vector=[0.01, 0.02, 0.03], payload={"agent_id": "agent-42", "content": "..."}),
])

results = client.query_points(
    collection_name="agent_memory",
    query=[0.01, 0.02, 0.03],
    query_filter=Filter(must=[FieldCondition(key="agent_id", match=MatchValue(value="agent-42"))]),
    limit=5,
).points

I ran this pattern (against Qdrant’s embedded local mode rather than a full server, which is enough to exercise the filtering and indexing logic) with three points across two tenant IDs, and the filtered query only ever came back with the matching tenant’s point. For a lot of small, similar tenants, this payload-partitioning approach is what Qdrant recommends by default. For a handful of large tenants that need stronger isolation, it also supports dedicated shards per tenant, and a tiered setup that starts tenants on a shared fallback shard and promotes the ones that grow past roughly 20,000 points to their own shard.

Here’s the gotcha that’ll bite anyone following an older tutorial: client.search(), the method every pre-2025 example uses, is gone in current qdrant-client. I tried calling it against a live client on version 1.19.1 and got AttributeError: 'QdrantClient' object has no attribute 'search', not a deprecation warning, a hard crash. query_points() is the replacement, and it’s what the code above uses. If you’re copying a Qdrant snippet from a two-year-old Stack Overflow answer, check for .search( first.

Dimension mismatches fail just as hard here, with a message that’s actually more specific than pgvector’s: inserting a 5-dimension vector into a collection configured for 8 dimensions threw ValueError: Wrong input: Vector dimension error: expected dim: 8, got 5 for vector '' in my test, caught before the point ever touched storage.

A single Qdrant collection holding separate tenant partitions for AI agent memory.

Chroma: Fastest to a Working Prototype, One Node Only

Chroma’s whole pitch is that you can go from zero to a working vector store in about four lines, no server to stand up, no schema to design upfront:

import chromadb

client = chromadb.PersistentClient(path="./agent_memory_db")
collection = client.create_collection(name="agent_memory", metadata={"hnsw:space": "cosine"})

collection.add(
    ids=["1", "2"],
    embeddings=[[0.01, 0.02, 0.03, 0.04], [0.05, 0.06, 0.07, 0.08]],
    metadatas=[{"agent_id": "agent-42"}, {"agent_id": "agent-7"}],
    documents=["User prefers terse answers", "User wants code examples only"],
)

results = collection.query(
    query_embeddings=[[0.015, 0.025, 0.035, 0.045]],
    n_results=5,
    where={"agent_id": "agent-42"},
)

That’s it. No server. No schema upfront. I ran this exact script locally with a persistent on-disk client, and the metadata filter worked immediately, no index tuning, no extension to enable.

This is genuinely why Chroma shows up in so many agent tutorials: there’s nothing standing between you and a working demo.

The honest limitation is right there in Chroma’s own system-constraints documentation: open-source, self-hosted Chroma is a single-node database. Scaling it means scaling the one node vertically, not adding replicas, and there’s no built-in replication or clustering in that deployment mode. The same docs note that in a single-node deployment, the HNSW index segment gets loaded into process memory on use, which means your available RAM is the practical ceiling on how many memories you can hold, not disk space. Chroma Cloud exists specifically to remove that ceiling, but at that point you’ve left the thing that made Chroma attractive in the first place: running it yourself, for free, with zero infrastructure.

Dimension errors here are the friendliest of the three, at least in wording: adding a 5-dimension embedding to a collection expecting 8 raised InvalidArgumentError: Collection expecting embedding with dimension of 8, got 5 when I tried it, which tells you immediately what’s wrong without digging through a stack trace.

I’d reach for Chroma specifically for a prototype, a side project, or an agent that will only ever run as one process against its own memory file. The instant you need two app servers writing to the same agent memory store concurrently, you’ve outgrown self-hosted Chroma’s actual deployment model, whatever the demo made it feel like.

Qdrant vs Chroma vs pgvector: Side by Side

AspectpgvectorQdrantChroma
What it isPostgres extension (vector column + index)Purpose-built vector databaseEmbedded/self-hosted vector store, Rust core
Multi-tenant patternA WHERE agent_id = ... clause like any SQL filterOne collection, payload field marked is_tenant: trueOne collection, where metadata filter
Index build costHNSW build wants to fit in maintenance_work_mem; slow, not silent, if it doesn’tCan disable the global index per tenant with payload_m for many small tenantsHNSW segment loads fully into process memory on use
Horizontal scaleWhatever your Postgres setup already does (read replicas, partitioning)Built-in sharding, dedicated or tiered per tenantVertical only, self-hosted; horizontal needs Chroma Cloud
Dimension-mismatch errorDataException: expected N dimensions, not MValueError: Vector dimension error: expected dim N, got MInvalidArgumentError: Collection expecting embedding with dimension of N, got M
Known gotchamaintenance_work_mem too small silently makes builds much slowerclient.search() is removed, not deprecated, as of recent qdrant-client versionsSelf-hosted mode has no replication or clustering, whatever the quickstart implies
Best fit for agent memoryYou already run Postgres for agent state or checkpointsMany agents/users, production, need real tenant isolationPrototype, single process, want zero infra

Pros of pgvector for Agent Memory

  • One database for agent state, checkpoints, and memory; one backup, one transaction boundary
  • Filtering is just SQL, so anyone who already knows your schema can query agent memory without learning a new client
  • HNSW index tuning (m, ef_construction, ef_search) is documented and standard Postgres index behavior, not a new mental model

Cons of pgvector for Agent Memory

  • You inherit Postgres’s own scaling story; there’s no built-in sharding designed around per-tenant vector workloads the way Qdrant has
  • Large HNSW builds need real maintenance_work_mem, and nobody tells you that until you read the fine print or watch the notice scroll by
  • No native multi-tenant isolation features beyond what you build yourself with a column and an index

Common Pitfalls

  • Creating one collection or table per tenant “to keep things clean.” It’s the first instinct and it’s wrong for all three stores once you pass a few dozen tenants; filter on a tenant field inside one collection instead.
  • Copying a Qdrant example that calls client.search(). On current qdrant-client that’s an AttributeError, not a warning. Use query_points().
  • Assuming self-hosted Chroma scales the way its quickstart feels. It’s single-node by design; plan for Chroma Cloud or a different store before you need horizontal scale, not after.
  • Building a large HNSW index on default Postgres memory settings and blaming pgvector for the slowness. Raise maintenance_work_mem first and watch for the notice telling you it’s too small.
  • Trusting your embedding pipeline’s dimension without a test. All three stores reject a mismatched vector outright, which is good, but you’ll only catch it before production if you actually try the wrong dimension once in a test.

FAQ

What’s the best vector database for AI agent memory?

There isn’t one universal answer. pgvector is the right call if you already run Postgres for agent state. Qdrant is the stronger choice once you have many agents or users and need real tenant isolation at scale. Chroma is the fastest way to get a single-process prototype working, with the clear limitation that self-hosted Chroma doesn’t scale horizontally on its own.

Can I use Chroma in production for an AI agent?

Yes, as long as production for you means a single node with enough RAM to hold the HNSW index, or you move to Chroma Cloud for managed horizontal scale. Self-hosted Chroma has no built-in replication or clustering. That’s the deployment model, by design, not an oversight.

Does pgvector support multi-tenant AI agent memory?

Yes, but not through a dedicated feature. You add an agent_id (or user_id) column, index it alongside your HNSW vector index, and filter on it in every query, same as you’d isolate any other tenant data in Postgres.

Why did my Qdrant code break with “object has no attribute ‘search'”?

You’re running a current qdrant-client version against code written for an older one. The search() method has been removed, not just deprecated; switch the call to query_points(), which takes the same vector and filter arguments.

Do I need a separate vector database if I’m already using semantic caching?

Not necessarily the same instance, but it’s worth being deliberate about it. If you’re already running semantic caching for LLM agents, that cache and your agent’s long-term memory store solve different problems (deduping repeated queries vs. recalling past context) and mixing them in one collection tends to make both harder to tune later.

Conclusion

If I’m starting a new agent today and I’m already running Postgres, I’m reaching for pgvector and not thinking twice about it; one less system to operate beats a marginally fancier index. If I’m building something with real multi-tenant scale in mind from day one, agents or users in the thousands, I’d go straight to Qdrant and set up payload partitioning before writing the first row, rather than discovering the one-collection-per-tenant trap the way I did.

Chroma still earns its spot for the first week of a project, when you just need the thing to work. Skip the debate about which store is “best” and ask a narrower question instead: will this still be one process, one node, six months from now? Answer that honestly, and the right store picks itself.

Your email address will not be published. Required fields are marked *