The Built-In Vector Store
The Azure AI Foundry Agent Service gives every agent a built-in Vector StoreA managed embedding database inside the Foundry project that stores chunked, vectorized document content. The Agent Service manages the underlying index automatically with no external infrastructure required. — a managed embedding database that sits inside the project and requires no external infrastructure. When you upload files to an agent, the service chunks them, generates embeddings, and stores those vectors so the File Search ToolThe built-in agent tool that performs semantic retrieval from the vector store (or a connected Azure AI Search index) and injects retrieved chunks into the model's context before generating a reply. can perform semantic retrieval at query time.
A vector store is a numeric representation of document content. Each chunk of text is converted into a high-dimensional embedding vector. At query time the user's question is embedded in the same space and the closest chunks are returned — this is Retrieval-Augmented Generation (RAG)A pattern where relevant document chunks are retrieved from a vector store and injected into the model's context to ground its response in specific source material.. You can tune retrieval precision by setting a Ranking ThresholdA minimum similarity score (0–1) that a chunk must meet to be included in File Search results. Setting this to 0.5–0.7 filters weakly matching chunks and reduces hallucination from tangentially related content. to filter out low-relevance chunks.
| Concept | Detail |
|---|---|
| Chunking | Files are split into overlapping text chunks (default ~800 tokens, ~400 overlap) |
| Embedding model | Managed by the service; you do not select or host it |
| Index scope | Per-agent; each agent has its own vector store namespace |
| Persistence | Files persist until explicitly deleted; re-upload is not needed across runs |
Supported File Types and Size Limits
| File Type | Extension | Notes |
|---|---|---|
.pdf | Text-layer PDFs only; scanned images are not OCR'd | |
| Word | .docx | Paragraph and table content extracted |
| Plain text | .txt | UTF-8 preferred |
| Markdown | .md | Headers and body extracted |
| PowerPoint | .pptx | Slide text extracted |
| HTML | .html | Tag content stripped |
| JSON / CSV | .json, .csv | Treated as structured text |
Size limits: Single file maximum is 512 MB. The total vector store per agent is limited to 100 GB. Each file is processed asynchronously — large files may take several minutes before they become searchable.
How the File Search Tool Uses the Vector Store
When a user sends a message, the File Search tool is invoked automatically (if enabled). The workflow is:
- The user query is embedded.
- The vector store returns the top-k most similar chunks (configurable via
max_num_results). - Retrieved chunks are injected into the system context before the model generates a reply.
- The response includes citations — inline references showing which file and chunk sourced each claim.
File Search Tool Configuration Parameters
| Parameter | Purpose | Default |
|---|---|---|
max_num_results | Maximum chunks returned per query | 20 |
ranking_threshold | Minimum similarity score (0–1) to include a chunk | 0 (all returned) |
chunk_overlap_tokens | Overlap between adjacent chunks (advanced) | 400 |
Setting ranking_threshold to 0.5–0.7 filters out weakly matching chunks and reduces hallucination risk from tangentially related content.
Vector Store vs Azure AI Search
| Criterion | Built-in Vector Store | Azure AI Search Connection |
|---|---|---|
| Setup effort | Zero — automatic | Requires a separate Azure AI Search resource |
| Data location | Managed inside the Foundry project | Your own subscription |
| Custom indexing | Not configurable | Full control (analyzers, fields, facets) |
| Scale | Up to 100 GB per agent | Unlimited (tiered pricing) |
| Hybrid search | Not supported | Supported (BM25 + vector) |
| Best for | Quick RAG on small–medium corpora | Enterprise indexes, compliance requirements |
Use the built-in vector store when you need rapid onboarding with minimal infrastructure. Choose Azure AI Search when you have an existing index, need hybrid retrieval, or must keep data in a specific region under your own tenant.
Hands-On: Upload a PDF and Confirm Indexing
Goal: Upload a PDF to an agent and confirm it is indexed and searchable.
- Navigate to Azure AI Foundry (https://ai.azure.com) and open your project.
- Click Agents in the left navigation, then select or create an agent.
- In the agent editor, click the Files tab.
- Click + Upload file → select a PDF from your local machine (under 512 MB).
- Watch the Status column — it transitions from Processing to Indexed (typically 10–60 seconds for small files).
- Switch to the Playground tab, type a question whose answer is in the PDF, and press Send.
- Confirm the response includes an inline citation with the file name and page reference.
- Expand the Tool calls panel to see the raw File Search input, output chunks, and similarity scores.
- (Optional) Go back to Files, click the file name, and inspect the Chunks list to see how the document was split.
- Try an out-of-scope question (not in the document) and verify the agent acknowledges it cannot answer rather than hallucinating.
AI-3018 Assessment Focus
File type limitations (especially scanned PDFs) and the distinction between vector store files and Code Interpreter files are the most common exam traps in this domain.
Exam Trap
"Upload via Azure Blob Storage to use File Search" — False. Files are uploaded directly through the Agent Service portal or SDK. You do not need a Storage account for the built-in vector store.
Exam Trap
"Scanned PDF images are automatically OCR'd" — False. The File Search tool extracts the text layer only. A scanned PDF with no embedded text produces no retrievable content.
Exam Trap
"The agent re-embeds files on every run" — False. Embeddings are computed once at upload time and stored persistently. They are not regenerated per conversation.
Exam Trap
"Setting max_num_results to 1 always improves accuracy" — Not necessarily. Too few chunks may omit the passage that answers the query; the model hallucinates to fill gaps.
Must Memorize
File Search files and Code Interpreter files are separate upload targets. A file uploaded to the vector store is NOT accessible to Code Interpreter, and vice versa.
Question — click to flip
Q: What happens when you upload a scanned PDF to the agent's vector store?
Question — click to flip
Q: What is the default maximum number of chunks returned by File Search per query?
Question — click to flip
Q: Are file embeddings recomputed at the start of each conversation?
Question — click to flip
Q: What is the single-file size limit for the built-in vector store?
Question — click to flip
Q: Which built-in vector store feature does Azure AI Search support that the Foundry vector store does not?
Question — click to flip
Q: What does setting ranking_threshold to 0.6 do?