Denis

I’m a passionate software engineer. Welcome to my blog!

By Denis Nuțiu

7 minute read

Introduction

Hello everyone! 👋

I wanted to write this article to show you how to build a RAG (Retrieval Augmented Generation) pipeline. One reason you may want to build it is to enable intent search for your business or project.

Sometimes RAG works best when paired with Lexical Search, the traditional search method where an inverted index is built and a ranking algorithm such as BM25+ is applied on the results.

The main benefit of using RAG is that agents can search huge amounts of data by intent, but humans can too. You can use it without agents as well.

RAG Diagram

The Project

In order to play around with this I wanted to index a book, and I’ve chosen the The Brothers Karamazov by Fyodor Dostoyevsky book and to my shame I haven’t read it.

Luckily I found it on Project Gutenberg and we can use its text for free.

You can download the text from the website and save it as a file, I also performed some additional cleaning by deleting the intro, outro and table of contents from the data.

Depending on the model used to generate embeddings, if it’s a multilingual model, users can ask questions about the data in virtually any language.

The Steps

To implement our RAG pipeline we’ll perform the following steps:

Rag Pipeline

To summarize the steps:

  1. We’re going to generate the embeddings using a open source python library called FlagEmbeddings and a model called “BAAI/bge-m3”
  2. We’re going to normalized the vectors using L2 norm.
  3. We’re going to reduce the dimensions from 1024 to 256. This is a tradeoff between accuracy and speed.
  4. We’re normalizing again.
  5. We store the final embeddings into a local vector database: chromadb.

When querying the RAG index, we need the same model to generate the embedding of the user’s query, and the PCA file to reduce the generated embedding dimensions to 256. Then we ask chromadb for nearest neighbours using the user’s query.

Generating the embeddings

I’ve used Python 3.14 and the following dependencies:

uv add FlagEmbeddinguv add scikit-learnuv add chromadb

The following Python code snippet builds the pipeline outlined above.

def main():    paragraphs = load_paragraphs(SOURCE_TEXT)    print(f"Loaded {len(paragraphs)} paragraphs from {SOURCE_TEXT}")    model = BGEM3FlagModel(EMBEDDING_MODEL, normalize_embeddings=False, use_fp16=True)    output = model.encode_corpus(paragraphs, batch_size=32, max_length=MAX_LENGTH)    embeddings = l2_normalize(output["dense_vecs"])    print(f"Embeddings shape: {embeddings.shape}")    pca = PCA(n_components=PCA_COMPONENTS)    embeddings = l2_normalize(pca.fit_transform(embeddings))    norms = np.linalg.norm(embeddings, axis=1)    print(        f"After PCA: shape {embeddings.shape}, "        f"explained variance {pca.explained_variance_ratio_.sum():.1%}, "        f"L2 norms in [{norms.min():.6f}, {norms.max():.6f}]"    )    np.savez(PCA_FILE, mean=pca.mean_, components=pca.components_)    print(f"Saved PCA projection to {PCA_FILE}")    # Rebuild the collection on every run so it always matches the PCA saved above.    client = chromadb.PersistentClient(path=str(CHROMA_PATH))    if COLLECTION_NAME in [collection.name for collection in client.list_collections()]:        client.delete_collection(COLLECTION_NAME)    # The vectors come from bge-m3 + PCA, so Chroma must not embed anything itself.    collection = client.create_collection(        COLLECTION_NAME, configuration={"hnsw": {"space": "cosine"}}, embedding_function=None    )    batch_size = client.get_max_batch_size()    for start in range(0, len(paragraphs), batch_size):        indices = range(start, min(start + batch_size, len(paragraphs)))        collection.add(            ids=[f"paragraph-{i}" for i in indices],            embeddings=embeddings[indices.start : indices.stop],            documents=paragraphs[indices.start : indices.stop],            metadatas=[{"paragraph": i} for i in indices],        )    print(f"Stored {collection.count()} paragraphs in collection {COLLECTION_NAME!r} at {CHROMA_PATH}")

Running the code will seed all embeddings into a local embedded chromadb vector database and save the PCA file on the disk. We’ve performed PCA on the embeddings to trade some accuracy for speed. Processing a 1024 vector requires more compute, we’ve applied a PCA to reduce its dimension to 256 elements.

Querying the embeddings

The user can query the embeddings via an MCP, web api or simple CLI. For now we use the CLI approach because we have the local database and pca file, which we need in order to build the querying vector.

def main() -> None:    parser = argparse.ArgumentParser(description="Search The Brothers Karamazov by meaning.")    parser.add_argument("query", help="the text to search for")    parser.add_argument("-k", "--top-k", type=int, default=5, help="number of paragraphs to return")    args = parser.parse_args()    client = chromadb.PersistentClient(path=str(CHROMA_PATH))    try:        collection = client.get_collection(COLLECTION_NAME, embedding_function=None)    except NotFoundError:        raise SystemExit(f"No collection {COLLECTION_NAME!r} in {CHROMA_PATH}; run main.py first")    pca = np.load(PCA_FILE)    model = BGEM3FlagModel(EMBEDDING_MODEL, normalize_embeddings=False, use_fp16=True)    query_vector = embed_query(args.query, model, pca["mean"], pca["components"])    results = collection.query(        query_embeddings=[query_vector],        n_results=args.top_k,        include=["documents", "metadatas", "distances"],    )    hits = zip(results["documents"][0], results["metadatas"][0], results["distances"][0])    for rank, (document, metadata, distance) in enumerate(hits, start=1):        # The collection uses cosine distance, so 1 - distance is the cosine similarity.        print(f"\n#{rank}  paragraph {metadata['paragraph']}  score {1 - distance:.3f}")        print(textwrap.fill(document, width=100))

We can query the database with:

uv run src/ragpipeline/query.py "Ivan's poem about the Grand Inquisitor"

It will create an embedding from the query, run the embedding through L2 norm, PCA, L2 and then query chroma db.

Indexing and query paths share the same PCA transform

The query will return:

#2  paragraph 1705  score 0.555“My poem is called ‘The Grand Inquisitor’; it’s a ridiculous thing, but I want to tell it to you.”#5  paragraph 5220  score 0.478“My dear fellow, I know a most charming and attractive young Russian gentleman, a young thinker anda great lover of literature and art, the author of a promising poem entitled _The Grand Inquisitor_.I was only thinking of him!”

Example queries

Here’s another query, but this time in Romanian, “l-au platit” meaning he was paid or bought.

uv run src/ragpipeline/query.py "L-au platit"#1  paragraph 630  score 0.492“Pay back the three thousand.”#2  paragraph 645  score 0.454“It was he told you about the money, then?”

We got some results involving money, which is pretty cool. And the last query in Romanian, “who are Fyodor’s partners?”.

uv run src/ragpipeline/query.py "Care sunt partenerele lui Fyodor?"#3  paragraph 1  score 0.393He was married twice, and had three sons, the eldest, Dmitri, by his first wife, and two, Ivan andAlexey, by his second. Fyodor Pavlovitch’s first wife, Adelaïda Ivanovna, belonged to a fairly richand distinguished noble family, also landowners in our district, the Miüsovs. How it came to passthat an heiress, who was also a beauty, and moreover one of those vigorous, intelligent girls, socommon in this generation, but sometimes also to be found in the last, could have married such aworthless, puny weakling, as we all called him, I won’t attempt to explain. I knew a young lady ofthe last “romantic” generation who after some years of an enigmatic passion for a gentleman, whomshe might quite easily have married at any moment, invented insuperable obstacles to their union,and ended by throwing herself one stormy night into a rather deep and rapid river from a high bank,almost a precipice, and so perished, entirely to satisfy her own caprice, and to be likeShakespeare’s Ophelia. Indeed, if this precipice, a chosen and favorite spot of hers, had been lesspicturesque, if there had been a prosaic flat bank in its place, most likely the suicide would neverhave taken place. This is a fact, and probably there have been not a few similar instances in thelast two or three generations. Adelaïda Ivanovna Miüsov’s action was similarly, no doubt, an echo ofother people’s ideas, and was due to the irritation caused by lack of mental freedom. She wanted,perhaps, to show her feminine independence, to override class distinctions and the despotism of herfamily. And a pliable imagination persuaded her, we must suppose, for a brief moment, that FyodorPavlovitch, in spite of his parasitic position, was one of the bold and ironical spirits of thatprogressive epoch, though he was, in fact, an ill-natured buffoon and nothing more. What gave themarriage piquancy was that it was preceded by an elopement, and this greatly captivated AdelaïdaIvanovna’s fancy. Fyodor Pavlovitch’s position at the time made him specially eager for any suchenterprise, for he was passionately anxious to make a career in one way or another. To attachhimself to a good family and obtain a dowry was an alluring prospect. As for mutual love it did notexist apparently, either in the bride or in him, in spite of Adelaïda Ivanovna’s beauty. This was,perhaps, a unique case of the kind in the life of Fyodor Pavlovitch, who was always of a voluptuoustemper, and ready to run after any petticoat on the slightest encouragement. She seems to have beenthe only woman who made no particular appeal to his senses.#4  paragraph 676  score 0.389Alyosha rose, but Fyodor Pavlovitch had already changed his mind.

Queries in different languages land near paragraphs with similar meaning

Agent Queries

And now for something different. We can tell the agent to query the RAG and note that this is quite efficient. The agent retrieves only the important stuff by doing semantic search instead of reading the entire book.

Retrieval, augmentation and generation with an agent

Claude Response to Query

Claude manages to identify Fyodor’s partners correctly.

Conclusion

Building a RAG pipeline is straightforward, in the age of AI you can ask your agent to do it for you but you can guide it. The FlagEmbedding python library contains sensible defaults and everything you need behind a simple and easy to use interface.

I’ve used a chromadb embedded local vector database for this project but you can also use other solutions, such as ElasticSearch, which gives you more features and lets you combine RAG with traditional search.

Thanks for reading!

The full code is available on my Github and Gitlab: