Getting started with LangChain

LangChain is a tool that helps in the creation of applications that want to fit into an existing Large language models (LLMs). While an LLM is good at answer a general question it may not be aware of the information, context, or behavior that your application would need.
LangChain helps us overcome this by preparing our question so that the LLM produces a viable answer.

A common preparation step in LangChain is:

You upload your content
LangChain splits this content into small documents or chunks
Embeds them into a vector database

When you provide the question:

The question is used to retrieve the most likeliest documents
Send the documents and the question to the LLM.
You receive an answer tailored to your data.

That is the steps in a nutshell. I will use be using TypeScript in this example.

Let's start by installing all the npm packages that we will be using in this example:

npm i dotenv pdf-parse ts-node typescript langchain @zilliz/milvus2-sdk-node

We will divide the application into 2 files.

load.ts, handles the preparation step.
index.ts, handles the question and answers.

Let us start with the load phase.

Load Phase

This phase refers to the following steps:

You upload your content
LangChain splits this content into small documents or chunks
Embeds them into a vector database

Preparing the data

First, create a folder in your working directory called /data. We will be placing .pdfs at this location. Place the 3 latest annual reports from Tesla in this location (2022-2020).

You can find teslas annual reports here: https://www.annreports.com/tesla

Preparing the vector store

We will be using Milvus as a vector store in the example. Milvus offers a convenient docker-compose file that can be downloaded from https://milvus.io. Even though I will provide it here, I really recommend you browse the latest version.

version: '3.5'

services:
  etcd:
    container_name: milvus-etcd
    image: quay.io/coreos/etcd:latest
    environment:
      - ETCD_AUTO_COMPACTION_MODE=revision
      - ETCD_AUTO_COMPACTION_RETENTION=1000
      - ETCD_QUOTA_BACKEND_BYTES=4294967296
      - ETCD_SNAPSHOT_COUNT=50000
    volumes:
      - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/etcd:/etcd
    command: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd

  minio:
    container_name: milvus-minio
    image: minio/minio:latest
    environment:
      MINIO_ACCESS_KEY: minioadmin
      MINIO_SECRET_KEY: minioadmin
    volumes:
      - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/minio:/minio_data
    command: minio server /minio_data
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  standalone:
    container_name: milvus-standalone
    image: milvusdb/milvus:latest
    command: ["milvus", "run", "standalone"]
    environment:
      ETCD_ENDPOINTS: etcd:2379
      MINIO_ADDRESS: minio:9000
    volumes:
      - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/milvus:/var/lib/milvus
    ports:
      - "19530:19530"
      - "9091:9091"
    depends_on:
      - "etcd"
      - "minio"

networks:
  default:
    name: milvus

Preparing env variables

We will be using dotenv in the example. Please make sure you have a .env file. In .env file need to have the following content:

OPENAI_API_KEY=[your-api-key]
MILVUS_URL=[your-milvus-url] //defaults to http://localhost:19530

Creating the load.ts script

We will start by making sure we include .env by performing:

import * as dotenv from 'dotenv';
dotenv.config();

Next, we will prepare the document loader to load the files that we want to embed into the vector store.

const loader = new DirectoryLoader('./data', {
    '.pdf': (path) => new PDFLoader(path),
  });

When prepared, we will use a text splitter to divide them into chunks.

 const docs = await loader.loadAndSplit(
    new RecursiveCharacterTextSplitter({ chunkSize: 500, chunkOverlap: 100 })
  );

We now have the docs. We will now use the Milvus client to upload the documents. Note that we are using OpenAIEmbeddings below. That is because we are going to use OpenAI's model. If you planning to use Huggingface or other models, you might require other embeddings.

 await Milvus.fromDocuments(docs, new OpenAIEmbeddings(), {
    collectionName: 'tesla_annual_reports',
  });

That´s it, we are now done with the load phase.

Question/answering phase

This step refers to:

The question is used to retrieve the most likeliest documents
Send the documents and the question to the LLM.
You receive an answer tailored to your data.

Next, we will prepare the model. We have a wide range of models to choose from and we can configure them individually. I will just use a default OpenAI model in this example.

const model = new OpenAI();

Next we will instantiate a new Milvus Client:

 const vectorStore = new Milvus(new OpenAIEmbeddings(), {
    collectionName: 'tesla_annual_reports',
  });

After that, we will prepare the chain. We will use the index-related chain called ConversationalRetrievalQAChain.

 const chain = ConversationalRetrievalQAChain.fromLLM(
    model,
    vectorStore.asRetriever()
  );

We now have everything set up in terms of LangChain to make the call.

const res = await chain.call({ question, chat_history: chatHistory });
console.log(res.text);

That's pretty much it. The res.text will contain the response. You can also choose to return the source documents which can help in verifying the response.

Test

Let's try it out:

Question:

What was teslas revenue for 2021?

Answer:

$53,823 million

Question:

What are the risks of investing in tesla based on the annual report of 2022?

Answer:

According to the risk factors section in Item 1A, risks of investing in Tesla include competition with other companies for talented employees, difficulty in managing business expansion, and the potential for technological innovations to not be successful.

Question:

Are there any additional risks?

Answer:

There is a risk that employees may leave Tesla or choose other employers over Tesla due to a competitive labor market. There is also a risk that issues with lithium-ion cells or other components manufactured at Tesla's Gigafactories could harm the production and profitability of Tesla's vehicle and energy storage products. There is also a risk that Elon Musk may have to sell shares of Tesla common stock to satisfy loan obligations or other commitments, which could cause the price of the stock to decline further. Finally, there is a risk that economic incentives may be implemented that provide benefits to domestic manufacturers that do not apply to Tesla, which could negatively impact demand for Tesla's vehicles.

Question:

Can you provide the sources for this?

Answer:

The sources regarding the risks of investing in Tesla based on the annual report of 2022 are the Risk Factors section (Item 1A) and the Unresolved Staff Comments section (Item 1B).

Conclusion

LangChain is a very convenient tool to use for adapting an application to work with an existing LLM. As we can see, not a lot of code is required to get something up and I think it can open up a lot of interesting opportunities.

I provide the complete source code below:

// load.ts
import { DirectoryLoader } from 'langchain/document_loaders/fs/directory';
import { PDFLoader } from 'langchain/document_loaders/fs/pdf';
import { Milvus } from 'langchain/vectorstores/milvus';
import { OpenAIEmbeddings } from 'langchain/embeddings/openai';
import { RecursiveCharacterTextSplitter } from 'langchain/text_splitter';

import * as dotenv from 'dotenv';
dotenv.config();

export const run = async () => {
  const loader = new DirectoryLoader('./data', {
    '.pdf': (path) => new PDFLoader(path),
  });

  const docs = await loader.loadAndSplit(
    new RecursiveCharacterTextSplitter({ chunkSize: 500, chunkOverlap: 100 })
  );

  const res = await Milvus.fromDocuments(docs, new OpenAIEmbeddings(), {
    collectionName: 'tesla_annual_reports',
  });

  console.log(res);
};

run();

//index.ts
import { OpenAI } from 'langchain/llms/openai';
import { ConversationalRetrievalQAChain } from 'langchain/chains';
import { OpenAIEmbeddings } from 'langchain/embeddings/openai';
import { Milvus } from 'langchain/vectorstores/milvus';
import readline from 'readline';

import * as dotenv from 'dotenv';
dotenv.config();

const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});

export const run = async () => {
  const model = new OpenAI();

  const vectorStore = new Milvus(new OpenAIEmbeddings(), {
    collectionName: 'tesla_annual_reports',
  });

  const chain = ConversationalRetrievalQAChain.fromLLM(
    model,
    vectorStore.asRetriever()
  );

  const chatHistory: string[] = [];
  while (true) {
    const question = await getUserInput(
      'Please enter your input or type "exit" to quit: '
    );

    if (!question) {
      break;
    }

    const res = await chain.call({ question, chat_history: chatHistory });
    console.log(res.text);
    chatHistory.push(question + res.text);
  }
};

const getUserInput = async (prompt: string): Promise<string | false> => {
  return new Promise((resolve) => {
    rl.question(prompt, (input) => {
      if (input === 'exit') {
        rl.close();
        resolve(false);
      } else {
        resolve(input);
      }
    });
  });
};

run();

Happy coding!

Rasmus Olsson