How to create a chatbot with OpenAI’s API

An architectural cheat sheet

I built a chatbot to help me probe long and tedious PDF documents, like my credit card contract!

So I’ve been trying to get up to speed on AI so I spent the last few days digesting content from all over the net.

I’m ready to get stuck in and build my first AI project, and one of those “ChatGPT but with your own documents” apps seems ideal.

So here’s what looks like the general architecture of such a chatbot app:

Setup

• Create an OpenAI account and get your API key.
• Create a vector database, e.g. PostgreSQL with pgvector or Redis.
• Create a table to hold all your embeddings (OpenAI’s vectors require 1536 dimensions).

Process your documents

• Gather all your documents ad convert them into text.
  – Convert line breaks and tabs to spaces (OpenAI’s recommendation).
• Break the document content into chunks.
  – Set it to something small-ish, e.g. 2,000 words (look up tokens for more specific settings).
• Use OpenAI’s API to create embeddings for each chunk.
• Save each chunk into the vector database.

Create a similarity search function

• Use a similarity search function to retrieve the correct chunk, based on a question.
  – OpenAI recommends the cosine similarity calculation.
• Test it out and tweak the parameters.

Create your user-facing app

• Create a form to ask questions and display responses.

Write your prompts

• Information gathering prompt: A prompt for OpenAI to keep asking questions until it has all the information it needs.
• Summarise question prompt: Another prompt to summarise the question.
• Answer prompt: A prompt that combines the final form of the question along with the relevant chunk of information.

Chat

• When someone asks a question, send the question and the Information gathering prompt to the OpenAI Chat API.
• Repeat until OpenAI responds that it has enough info.
• Send the chat transcript to OpenAI with the Summarise question prompt.

Answer question

• Use the OpenAI API to convert the summarised question into an embedding.
• Search the vector database for the chunk that would most likely contain the answer, using the similarity search function you created earlier.
• Use the Answer prompt to send the summarised question and the chunk to the OpenAI Completion API to get the answer.
• Wait for OpenAI to respond and display the answer to the user.

I’ll be sharing more of these how-tos and guides. If you don’t want to miss it, you can subscribe to my newsletter.

Want more? > Follow along as I build my first AI product