Home All posts

dotnet ai architecture

This post is based on my talk at Dotnet Georgia.

I wanted to turn that talk into a written version. The focus is on what RAG is and why it matters. I will also show how to build it in .NET with Microsoft.Extensions.VectorData.

Even the smartest AI models do not know who Lacrimosa is. She is my cat. But once I give the model access to my own data through a RAG pipeline, it can answer that question correctly. RAGrimosa is a small .NET 10 sample that shows the complete path from a local text file to a grounded answer.

LLMs Are Useful but Not Reliable by Default

LLMs changed how we build software. They are good at working with language. They can summarize information and generate code. They can also turn rough input into something structured.

That part is real. The part people often skip is the limitation.

An LLM is not a live system of record. It is a model trained on past data. The answer can sound confident while being outdated or incomplete. It can also be wrong.

In practice, the usual failure modes look like this:

This is where patterns like RAG become useful. Instead of asking the model to answer from training data alone, we give it relevant context at request time.

Before getting into RAG, it helps to cover the building blocks behind it.

Vectors

A vector is just a list of numbers. In AI systems, those numbers represent some features of the original data. For text, that usually means semantic meaning rather than exact wording.

The important idea is distance. If two vectors are close to each other, the source data is usually similar as well.

Embeddings

Embeddings are vectors created from real-world data. This can be text code images or audio. They capture meaning in a way that makes similarity search possible.

For example, two sentences can use different words and still end up with similar embeddings if they mean roughly the same thing.

Vector stores

Vector stores are databases built for this kind of search. They let you store embeddings and retrieve the nearest matches quickly.

That matters because RAG depends on finding relevant context fast enough to use it during a request. Instead of matching exact words, you search by meaning.

RAG

RAG stands for Retrieval-Augmented Generation.

The idea is simple. Before the model answers the system retrieves relevant information from an external source. The source can be a document database or internal knowledge base. That context is then included in the prompt.

A typical flow looks like this:

RAG does not make the model smarter in a general sense. It makes the answer more grounded in data you control.

RAG Diagram

RAG matters because it solves a very practical problem: most real systems need answers based on current, domain-specific data.

It also gives you a cleaner operating model:

That does not remove every failure mode. It is still a better foundation than hoping the model “just knows” your business context.

Microsoft.Extensions.VectorData

Microsoft.Extensions.VectorData is a .NET library that gives you a consistent abstraction over vector stores.

It feels familiar from a .NET developer’s perspective. You work with collections records and attributes instead of wiring every provider differently. This makes early experiments easier. It also keeps the code cleaner as the solution grows.

Core components

The main pieces are straightforward.

VectorStore can be used as the entry point when an application needs to work with multiple collections. RAGrimosa keeps the example smaller and injects a typed collection directly.

VectorStoreCollection<TKey, TRecord> represents a concrete collection of records. A record usually has an ID with some metadata and one or more vector fields.

The model is shaped with attributes:

The library stays close to the underlying concepts and removes repetitive plumbing. In RAGrimosa the vector property returns the chunk text. The registered IEmbeddingGenerator turns it into an embedding during upsert. It also embeds the user question during search.

If you are building RAG in .NET, that is a good tradeoff.

How RAGrimosa is structured

The sample has two distinct phases.

During ingestion it:

  1. reads RAGrimosa/data/source.txt
  2. splits the text into overlapping character-based chunks
  3. gives every chunk a deterministic ID
  4. creates the PostgreSQL collection if needed
  5. upserts the chunks while the embedding generator creates their vectors

During question answering it:

  1. embeds the user’s question and searches for the five closest chunks by default
  2. formats those chunks into a numbered context block
  3. sends the system prompt with the retrieved context and question to the chat model
  4. prints the answer followed by the source chunk IDs and similarity scores

The application uses the .NET generic host with options validation dependency injection and structured logging. Program.cs builds the host and handles Ctrl+C. It then resolves RagOrchestrator and starts the workflow. The implementation stays in the ingestion and orchestration services.

Running the sample locally

I put together a small reference project for this post: RAGrimosa.

It is intentionally simple but still runs the complete pipeline:

The fastest way to run it is with Docker Compose.

The main settings live in RAGrimosa/appsettings.json:

{
  "OpenAI": {
    "ApiKey": "",
    "ChatModel": "gpt-4o-mini",
    "EmbeddingModel": "text-embedding-3-small"
  },
  "Postgres": {
    "ConnectionString": "Host=db;Database=rag_test;Port=5432;Username=postgres;Password=postgrespw",
    "CollectionName": "documents"
  },
  "Ingestion": {
    "InputFilePath": "data/source.txt",
    "ChunkSize": 1200,
    "ChunkOverlap": 150,
    "RecreateCollection": true
  },
  "Rag": {
    "SearchResultCount": 5,
    "SystemPrompt": "You are a helpful research assistant. Answer questions using the provided context snippets and focus on clarity. Do not add citation markers in the response."
  }
}

The option classes validate these values when the application starts. A missing API key or connection string fails early instead of much later in the RAG flow.

Option 1: run everything with Docker

Prerequisites:

Then:

git clone https://github.com/gabisonia/RAGrimosa.git
cd RAGrimosa

Set your OpenAI API key. You can put it in the OpenAI section of RAGrimosa/appsettings.json as shown in the repository README. You can also export it to avoid saving the secret in a file:

export OpenAI__ApiKey="your-api-key"

Then start the app:

docker compose run --rm --build -e OpenAI__ApiKey app

The Compose service waits for PostgreSQL to become healthy. The initialization script enables the vector extension. The app then runs ingestion and opens an interactive prompt:

user >

At that point you can start asking questions about the ingested file.

Option 2: run Postgres in Docker but run the app with dotnet

If you want the database in a container but the app on your machine, use this flow instead.

Prerequisites:

First start only the database:

docker compose up -d db

Then override the database host and supply the API key without changing the checked-in configuration:

export Postgres__ConnectionString="Host=localhost;Database=rag_test;Port=5432;Username=postgres;Password=postgrespw"
export OpenAI__ApiKey="your-api-key"

After that, run the console app:

dotnet run --project RAGrimosa/RAGrimosa.csproj

The default setup is small on purpose. That makes it easy to see the RAG pipeline end to end without too much framework noise.

Submit an empty line or press Ctrl+C to stop the app. When you are finished remove the Compose resources with docker compose down. The named PostgreSQL volume remains. With RecreateCollection set to true the sample recreates the vector collection on its next startup.

What the code looks like

The full repo is here:

1. Define the record that goes into the vector store

The DocumentChunk model is where Microsoft.Extensions.VectorData starts to feel useful. The record is plain C# and the attributes make the intent clear:

internal sealed class DocumentChunk
{
    [VectorStoreKey]
    public required string Id { get; init; }

    [VectorStoreData]
    public required string Content { get; init; }

    [VectorStoreData]
    public required string Source { get; init; }

    [VectorStoreData]
    public int ChunkIndex { get; init; }

    [VectorStoreVector(Dimensions: 1536, DistanceFunction = DistanceFunction.CosineSimilarity)]
    public string Embedding => Content;
}

The data model stays readable. Id is the collection key. Content Source and ChunkIndex are stored metadata. Embedding is the searchable vector field.

The interesting detail is that Embedding is a string and not a ReadOnlyMemory<float>. Returning Content tells the vector data pipeline which text to send to the embedding generator. The 1,536 dimensions match the default text-embedding-3-small model. Cosine similarity controls how results are ranked. If the embedding model has a different output size then this dimension must change as well.

2. Register the chat client embedding client and vector collection

The project uses a standard host builder and wires everything in one place:

builder.Services.AddEmbeddingGenerator(sp =>
{
    var options = sp.GetRequiredService<IOptions<OpenAiOptions>>().Value;
    return new EmbeddingClient(options.EmbeddingModel, options.ApiKey).AsIEmbeddingGenerator();
});

builder.Services.AddChatClient(sp =>
{
    var options = sp.GetRequiredService<IOptions<OpenAiOptions>>().Value;
    return new ChatClient(options.ChatModel, options.ApiKey).AsIChatClient();
});

builder.Services.AddPostgresCollection<string, DocumentChunk>(
    postgresConfiguration.CollectionName,
    postgresConfiguration.ConnectionString);

That is the core of the setup. There is one client for embeddings and another for chat. The typed collection is backed by PostgreSQL. The application code depends on IEmbeddingGenerator IChatClient and VectorStoreCollection<string, DocumentChunk>. It does not depend directly on the OpenAI or pgvector clients.

3. Ingest the source document as chunks

The ingestion service reads the local file and splits it into overlapping chunks. It then upserts them into the collection:

var fileContent = await File.ReadAllTextAsync(ingestionOptions.InputFilePath, cancellationToken);
var chunks = SplitIntoChunks(fileContent, ingestionOptions.ChunkSize, ingestionOptions.ChunkOverlap);

var sourceName = Path.GetFileName(ingestionOptions.InputFilePath);
var records = new List<DocumentChunk>(chunks.Count);
for (var index = 0; index < chunks.Count; index++)
{
    records.Add(new DocumentChunk
    {
        Id = CreateStableChunkId(sourceName, index),
        Content = chunks[index],
        Source = sourceName,
        ChunkIndex = index,
    });
}

await collection.UpsertAsync(records, cancellationToken: cancellationToken);

This is the part many RAG demos skip over too quickly. Chunking strategy matters. File boundaries matter. Stable IDs matter if you want re-ingestion to behave predictably.

The default chunk size is 1,200 characters with an overlap of 150. IDs use the normalized file name and a zero-padded chunk index such as source-0000. Ingesting the same source again updates the existing records instead of producing duplicates.

4. Retrieve context and build the grounded prompt

Once ingestion is done, the orchestrator searches the collection and sends the retrieved snippets to the chat model:

await foreach (var result in collection.SearchAsync(query, top, cancellationToken: cancellationToken))
{
    results.Add(result);
}

var chatMessages = new[]
{
    new ChatMessage(ChatRole.System, ragSettings.SystemPrompt),
    new ChatMessage(ChatRole.User, $"{BuildContextSection(searchResults)}Question:\n{question}"),
};

var response = await chatClient.GetResponseAsync(chatMessages, cancellationToken: cancellationToken);

This is the actual RAG loop in a small amount of code:

The model answers without inline citation markers. The console prints the context chunks separately. This makes retrieval visible while you experiment with chunk size overlap and result count.

After asking a question about Lacrimosa the output ends with entries in this shape:

Context chunks:
[1] source.txt chunk 2 score=0.842
[2] source.txt chunk 0 score=0.801

The scores above are illustrative; the actual values depend on the query and embedding model.

RAG solves a normal engineering problem. The model does not know your private context or your documents. It also does not know facts like who Lacrimosa is. You have to provide that context in a repeatable way.

A small console app with Postgres pgvector and Microsoft.Extensions.VectorData is enough to show the idea. It only has the core flow from ingestion to retrieval and then to the answer.

Source Code