Create your own ChatGPT in Dart with Supabase Vector and OpenAI

Create your own ChatGPT in Dart with Supabase Vector and OpenAI

Christos's photo
Christos
·

7 min read

Introduction

ChatGPT and AI-powered tools have been increasingly popular these days. Most recently, a lot of websites, apps and services have integrated ChatGPT or ChatGPT-like tools so you can "chat" with their documentation and content, most notably Supabase Clippy.

In this tutorial, we will build our own ChatGPT, with which we'll be able to store any documents in text format, and ask it questions about these documents. And while Python and JavaScript/TypeScript have been the most popular choices for such tools, we'll be doing this in Dart!

Why Dart? Well, this is a Dart and Flutter blog after all. For a Flutter app with AI capabilities, the AI backend could be written in any language. But why not Dart? It makes things easier as the full stack is written in the same language, and models such as requests and responses can be reused. And while some could argue that the Dart libraries are not as advanced, for this ChatGPT clone we only really need two things: Supabase Vector and OpenAI, both of which have available Dart packages.

Setting up the project

To keep things simple, we'll build our ChatGPT clone as a simple API with two endpoints: one for "embedding" new information, where we upload any documents/text we would like to ask questions on, and a chat endpoint to ask questions and get answers based on the uploaded documents.

We will use Dart and the shelf package for the API. Supabase will be used to store the documents as well as their embeddings, which will be used to perform similarity search thanks to Supabase's pgvector plugin.

Let's create our new Dart project:

dart create dart_ai_api
cd dart_ai_api

Our first dependencies to build out the API are shelf, shelf_router and dart_openai:

dart pub add shelf
dart pub add shelf_router
dart pub add dart_openai

Next, we're adding Supabase as a dependency, but also initializing a Supabase project within our Dart project.

dart pub add supabase
supabase init

For additional information on the Supabase project initialization, check out the documentation here.

Supabase Vector

Supabase Vector is actually a set of tools to help you build AI apps, and what we'll actually be using for this project is pgvector, a Postgres extension which allows you to store and query vectors in Postgres. And if you're new to Supabase, the Supabase service includes a Postgres database!

Preparing the database

In the seed file, we will create a table to store documents and embeddings, as well as a function to perform similarity search. First, we need to enable the pgvector plugin.

-- supabase/seed.sql
create extension vector with schema extensions;

Next, the documents table:

-- supabase/seed.sql
create table documents (
  id serial primary key,
  title text not null,
  body text not null,
  embedding vector(1536)
);

And last but not least, the similarity search function:

-- supabase/seed.sql
create or replace function match_documents (
  query_embedding vector(1536),
  match_threshold float,
  match_count int
)
returns table (
  id bigint,
  body text,
  similarity float
)
language sql stable
as $$
  select
    documents.id,
    documents.body,
    1 - (documents.embedding <=> query_embedding) as similarity
  from documents
  where 1 - (documents.embedding <=> query_embedding) > match_threshold
  order by similarity desc
  limit match_count;
$$;

There is a lot to explain, but the documentation over at Supabase does a good job introducing all the concepts. So I won't be going too in-depth on the details and rather focusing on getting this to work in Dart.

At this point we could run our Supabase project locally, which will create the database including the documents table and function.

supabase start

The Dart API

Let's build the Dart API next. We'll start with an AI service class, which will handle the interactions with Supabase and OpenAI. This service will then be called by the endpoint handlers.

class AIService {
  static const _openAiKey = "SECRET";
  static const _supabaseUrl = "http://localhost:54321";
  static const _supabaseKey = "SECRET";

  final OpenAI openAi;
  final SupabaseClient supabase;

  factory AIService.init() {
    OpenAI.apiKey = _openAiKey;
    final supabase = SupabaseClient(_supabaseUrl, _supabaseKey);
    return AIService(OpenAI.instance, supabase);
  }

  AIService(this.openAi, this.supabase);
}

The embed method will receive a document's title and body, and store this data along with the embeddings created by calling the OpenAI API. For the embedding model we use text-embedding-ada-002, which is recommended by OpenAI for most use cases. Note that we should use the same model for all embeddings, otherwise similarity search will not work.

void embed(EmbedRequest request) async {
  final response = await openAi.embedding.create(
    model: "text-embedding-ada-002",
    input: request.body,
  );
  final embedding = response.data.first.embeddings;
  await supabase.from("documents").insert({
    "title": request.title,
    "body": request.body,
    "embedding": embedding,
  });
}

The second and last method of our AI service is the chat functionality. It does several things:

  • Creates an embedding of the question we pass to it, similar to the embed functionality previously.
  • Uses that embedding to perform similarity search on our existing saved documents, using the function we created during the database setup step.
  • Calls the OpenAI Chat API with a system prompt including any relevant documents found as the context, as well as the question.
Future<String> chat(ChatRequest request) async {
  // create an embedding of the message to perform similarity search
  final embeddingResponse = await openAi.embedding.create(
    model: "text-embedding-ada-002",
    input: request.message,
  );
  final embedding = embeddingResponse.data.first.embeddings;

  // retrieve up to 5 most similar documents to include in chat system prompt
  final results = await supabase.rpc('match_documents', params: {
    'query_embedding': embedding,
    'match_threshold': 0.8,
    'match_count': 5,
  });

  // combine document content together
  var context = '';
  for (var document in results) {
    document as Map<String, dynamic>;
    final content = document['body'] as String;
    context += '$content\n---\n';
  }

  final prompt = """
      You are a helpful AI assistant.
      Given the following sections, answer any user questions by
      using only that information.
      If you are unsure and the answer is not explicitly written in
      the sections below, say "Sorry, I can't help you with that."

      Context sections:
      $context
    """;

  final chatResponse =
      await openAi.chat.create(model: 'gpt-3.5-turbo', messages: [
    OpenAIChatCompletionChoiceMessageModel(
        role: OpenAIChatMessageRole.system, content: prompt),
    OpenAIChatCompletionChoiceMessageModel(
        role: OpenAIChatMessageRole.user, content: request.message),
  ]);
  return chatResponse.choices.first.message.content;
}

A request to OpenAI's Chat API has a maximum token limit. We cannot just give it all our documents as the context in the prompt. Because of this, similarity search through vector embeddings is a solution for this. Of course, this depends on the total number of documents, and if there's only a few, this could still be possible. Additionally, the more tokens that are part of the request, the more expensive the API call.

In the chat method above, you can see from the RPC call to Supabase that we are only returning the 5 most relevant documents that are past the threshold. Both the threshold and count parameters can be tweaked as desired. By only passing the few most relevant documents to OpenAI, we can get a better, faster, and cheaper response.

API routes

The last piece is the router which route the API requests to the appropriate AI service method. The shelf and shelf_router packages make this quite easy.

void run() async {
  final app = Router();
  final ai = AIService.init();

  app.post('/embed', (Request request) async {
    try {
      final json = await _requestToJson(request);
      ai.embed(EmbedRequest.fromJson(json));
      return Response.ok('embedding saved');
    } catch (err) {
      print(err);
      return Response.internalServerError(body: 'something went wrong');
    }
  });

  app.post('/chat', (Request request) async {
    try {
      final json = await _requestToJson(request);
      final response = await ai.chat(ChatRequest.fromJson(json));
      return Response.ok(response);
    } catch (err) {
      print(err);
      return Response.internalServerError(body: 'something went wrong');
    }
  });

  final server = await io.serve(app, 'localhost', 8080);
  print('Server running on localhost:${server.port}');
}

Future<Map<String, dynamic>> _requestToJson(Request request) async {
  final reqString = await request.readAsString();
  return jsonDecode(reqString);
}

After this, we are now ready to run our Dart server. Make sure to start Supabase as well if not already running.

supabase start
dart run

The API in action

We can now store documents by calling the /embed endpoint:

curl --header "Content-Type: application/json" \
  --request POST \
  --data '{"title":"Flutter","body":"Flutter is an open-source, cross-platform UI software development kit"}' \
  http://localhost:8080/embed

And ask questions by calling the /chat endpoint:

curl --header "Content-Type: application/json" \
  --request POST \
  --data '{"message":"What is Flutter?"}' \
  http://localhost:8080/chat

The answer to the above question should be taken from the previous data we stored! We can now use this API to store any kind of information we want, and ask questions about it.

Wrapping up

In this post, we built our own ChatGPT API which we can use to store textual data and ask questions on it, all in Dart, and with the help of Supabase and the OpenAI API. You can find the full source code here.

Did you find this article valuable?

Support Christos by becoming a sponsor. Any amount is appreciated!

DartAIchatgptsupabase