# Create your own ChatGPT in Dart with Supabase Vector and OpenAI ## Introduction ChatGPT and AI-powered tools have been increasingly popular these days. Most recently, a lot of websites, apps and services have integrated ChatGPT or ChatGPT-like tools so you can "chat" with their documentation and content, most notably [Supabase Clippy](https://supabase.com/blog/chatgpt-supabase-docs). In this tutorial, we will build our own ChatGPT, with which we'll be able to store any documents in text format, and ask it questions about these documents. And while Python and JavaScript/TypeScript have been the most popular choices for such tools, we'll be doing this in Dart! Why Dart? Well, this is a Dart and Flutter blog after all. For a Flutter app with AI capabilities, the AI backend could be written in any language. But why not Dart? It makes things easier as the full stack is written in the same language, and models such as requests and responses can be reused. And while some could argue that the Dart libraries are not as advanced, for this ChatGPT clone we only really need two things: [Supabase Vector](https://supabase.com/docs/guides/ai/vector-columns) and OpenAI, both of which have available Dart packages. ## Setting up the project To keep things simple, we'll build our ChatGPT clone as a simple API with two endpoints: one for "embedding" new information, where we upload any documents/text we would like to ask questions on, and a chat endpoint to ask questions and get answers based on the uploaded documents. We will use Dart and the `shelf` package for the API. Supabase will be used to store the documents as well as their [embeddings](https://platform.openai.com/docs/guides/embeddings/what-are-embeddings), which will be used to perform similarity search thanks to Supabase's `pgvector` plugin. Let's create our new Dart project: ```sh dart create dart_ai_api cd dart_ai_api ``` Our first dependencies to build out the API are `shelf`, `shelf_router` and `dart_openai`: ```sh dart pub add shelf dart pub add shelf_router dart pub add dart_openai ``` Next, we're adding Supabase as a dependency, but also initializing a Supabase project within our Dart project. ```sh dart pub add supabase supabase init ``` For additional information on the Supabase project initialization, check out the documentation [here](https://supabase.com/docs/reference/cli/supabase-init). ## Supabase Vector Supabase Vector is actually a set of tools to help you build AI apps, and what we'll actually be using for this project is `pgvector`, a Postgres extension which allows you to store and query vectors in Postgres. And if you're new to Supabase, the Supabase service includes a Postgres database! ### Preparing the database In the seed file, we will create a table to store documents and embeddings, as well as a function to perform similarity search. First, we need to enable the `pgvector` plugin. ```sql -- supabase/seed.sql create extension vector with schema extensions; ``` Next, the documents table: ```sql -- supabase/seed.sql create table documents ( id serial primary key, title text not null, body text not null, embedding vector(1536) ); ``` And last but not least, the similarity search function: ```sql -- supabase/seed.sql create or replace function match_documents ( query_embedding vector(1536), match_threshold float, match_count int ) returns table ( id bigint, body text, similarity float ) language sql stable as $$ select documents.id, documents.body, 1 - (documents.embedding <=> query_embedding) as similarity from documents where 1 - (documents.embedding <=> query_embedding) > match_threshold order by similarity desc limit match_count; $$; ``` There is a lot to explain, but the [documentation](https://supabase.com/docs/guides/ai/vector-columns) over at Supabase does a good job introducing all the concepts. So I won't be going too in-depth on the details and rather focusing on getting this to work in Dart. At this point we could run our Supabase project locally, which will create the database including the documents table and function. ```sh supabase start ``` ## The Dart API Let's build the Dart API next. We'll start with an AI service class, which will handle the interactions with Supabase and OpenAI. This service will then be called by the endpoint handlers. ```dart class AIService { static const _openAiKey = "SECRET"; static const _supabaseUrl = "http://localhost:54321"; static const _supabaseKey = "SECRET"; final OpenAI openAi; final SupabaseClient supabase; factory AIService.init() { OpenAI.apiKey = _openAiKey; final supabase = SupabaseClient(_supabaseUrl, _supabaseKey); return AIService(OpenAI.instance, supabase); } AIService(this.openAi, this.supabase); } ``` The embed method will receive a document's title and body, and store this data along with the embeddings created by calling the OpenAI API. For the embedding model we use `text-embedding-ada-002`, which is recommended by OpenAI for most use cases. Note that we should use the same model for all embeddings, otherwise similarity search will not work. ```dart void embed(EmbedRequest request) async { final response = await openAi.embedding.create( model: "text-embedding-ada-002", input: request.body, ); final embedding = response.data.first.embeddings; await supabase.from("documents").insert({ "title": request.title, "body": request.body, "embedding": embedding, }); } ``` The second and last method of our AI service is the chat functionality. It does several things: * Creates an embedding of the question we pass to it, similar to the embed functionality previously. * Uses that embedding to perform similarity search on our existing saved documents, using the function we created during the database setup step. * Calls the OpenAI Chat API with a system prompt including any relevant documents found as the context, as well as the question. ```dart Future chat(ChatRequest request) async { // create an embedding of the message to perform similarity search final embeddingResponse = await openAi.embedding.create( model: "text-embedding-ada-002", input: request.message, ); final embedding = embeddingResponse.data.first.embeddings; // retrieve up to 5 most similar documents to include in chat system prompt final results = await supabase.rpc('match_documents', params: { 'query_embedding': embedding, 'match_threshold': 0.8, 'match_count': 5, }); // combine document content together var context = ''; for (var document in results) { document as Map; final content = document['body'] as String; context += '$content\n---\n'; } final prompt = """ You are a helpful AI assistant. Given the following sections, answer any user questions by using only that information. If you are unsure and the answer is not explicitly written in the sections below, say "Sorry, I can't help you with that." Context sections: $context """; final chatResponse = await openAi.chat.create(model: 'gpt-3.5-turbo', messages: [ OpenAIChatCompletionChoiceMessageModel( role: OpenAIChatMessageRole.system, content: prompt), OpenAIChatCompletionChoiceMessageModel( role: OpenAIChatMessageRole.user, content: request.message), ]); return chatResponse.choices.first.message.content; } ``` ### Why similarity search? A request to OpenAI's Chat API has a maximum token limit. We cannot just give it all our documents as the context in the prompt. Because of this, similarity search through vector embeddings is a solution for this. Of course, this depends on the total number of documents, and if there's only a few, this could still be possible. Additionally, the more tokens that are part of the request, the more expensive the API call. In the `chat` method above, you can see from the RPC call to Supabase that we are only returning the 5 most relevant documents that are past the threshold. Both the threshold and count parameters can be tweaked as desired. By only passing the few most relevant documents to OpenAI, we can get a better, faster, and cheaper response. ## API routes The last piece is the router which route the API requests to the appropriate AI service method. The `shelf` and `shelf_router` packages make this quite easy. ```dart void run() async { final app = Router(); final ai = AIService.init(); app.post('/embed', (Request request) async { try { final json = await _requestToJson(request); ai.embed(EmbedRequest.fromJson(json)); return Response.ok('embedding saved'); } catch (err) { print(err); return Response.internalServerError(body: 'something went wrong'); } }); app.post('/chat', (Request request) async { try { final json = await _requestToJson(request); final response = await ai.chat(ChatRequest.fromJson(json)); return Response.ok(response); } catch (err) { print(err); return Response.internalServerError(body: 'something went wrong'); } }); final server = await io.serve(app, 'localhost', 8080); print('Server running on localhost:${server.port}'); } Future> _requestToJson(Request request) async { final reqString = await request.readAsString(); return jsonDecode(reqString); } ``` After this, we are now ready to run our Dart server. Make sure to start Supabase as well if not already running. ```sh supabase start dart run ``` ## The API in action We can now store documents by calling the `/embed` endpoint: ```sh curl --header "Content-Type: application/json" \ --request POST \ --data '{"title":"Flutter","body":"Flutter is an open-source, cross-platform UI software development kit"}' \ http://localhost:8080/embed ``` And ask questions by calling the `/chat` endpoint: ```sh curl --header "Content-Type: application/json" \ --request POST \ --data '{"message":"What is Flutter?"}' \ http://localhost:8080/chat ``` The answer to the above question should be taken from the previous data we stored! We can now use this API to store any kind of information we want, and ask questions about it. ## Wrapping up In this post, we built our own ChatGPT API which we can use to store textual data and ask questions on it, all in Dart, and with the help of Supabase and the OpenAI API. You can find the full source code [here](https://github.com/dartling/dart_api_api).