Next.js Course: Generating and streaming text with ChatGPT

In this lesson, we learn how to generate posts with OpenAI and to display them using a table

In the previous section, we learned how to set up a schema with Supabase, and added two tables: users and posts.

In this section, we will learn how to generate blog posts with OpenAI's ChatGPT and insert them into the posts table.

To do so, we will:

Create a form to collect user input for the blog post - i.e. such as the topic
Use OpenAI's ChatGPT API to generate the blog post content and stream text from the API in real-time.
Finally, insert the posts into the posts table once generated by ChatGPT.

In the next sections, we will learn how to display the data from the posts table on the front end.

Installing the OpenAI SDKs

Open AI SDK: We need to install the package openai.
Vercel AI SDK: Additionally, we install the package ai from Vercel which makes streaming text from the API seamless and easy.

Run the following command to install the packages:

npm i ai openai

In a previous version, we used to use the library openai-edge. This is no longer needed because the OpenAI SDK v4 supports edge environments.

Creating the OpenAI Client

Now that the packages are installed, we can create the OpenAI client.

We will create a file at lib/openai-client.ts and insert the following code:

lib/openai-client.ts

import OpenAI from 'openai';
const apiKey = process.env.OPENAI_API_KEY;
if (!apiKey) {
  throw new Error('OPENAI_API_KEY env variable is not set');
}
function getOpenAIClient() {
  return new OpenAI({ apiKey });
}
export default getOpenAIClient;

Now we can easily use the client in our server actions by importing the getOpenAIClient function:

import getOpenAIClient from '@/lib/openai-client';
const client = getOpenAIClient();

Adding the OPENAI_API_KEY to the environment variables

To add the OPENAI_API_KEY to the environment variables, we need to add it to the .env.local file (create it if it doesn't exist):

.env.local

OPENAI_API_KEY=***************************************

NB: replace the * with your OpenAI API key.

If you don't have an OpenAI API key, you can get one by signing up to OpenAI.

Updating the Dashboard page to redirect to the form

Since we need users to navigate to the form to generate a new post, we need to update the dashboard page to redirect to the form.

We will do so by adding a button to the dashboard page with the label Create New Post. The link will redirect users to the /new page, which will display the form.

app/(app)/dashboard/page.tsx

import Link from "next/link";
import { LucideLayoutDashboard } from "lucide-react";
import { Button } from "@/components/ui/button";
function DashboardPage() {
  return (
    <div className='container'>
      <div className='flex flex-col flex-1 space-y-8'>
        <div className='flex justify-between items-start'>
          <h1 className='text-2xl font-semibold flex space-x-4 items-center'>
            <LucideLayoutDashboard className='w-5 h-5' />
            <span>
              Dashboard
            </span>
          </h1>
          <Button>
            <Link href='/new'>
              Create New Post
            </Link>
          </Button>
        </div>
      </div>
    </div>
  );
}
export default DashboardPage;

From here, we can navigate to the /new page, which we will create in the next section.

Since we haven't created the /new page yet, we will get a 404 error. We will create the page in the next section.

Streaming Text with the ChatGPT API

To generate a blog post, we need to:

Titles suggestions: Ask the user the topic they want to write about
Post content: Generate the blog post content using the topic

In this section, we will learn two fundamental concepts for any AI SaaS application:

Text Streaming - Streaming text using a Route Handler API and the ai package from Vercel
Content Generation - Using Server Actions to request the ChatGPT API and generate the blog post content

Creating a Route API for streaming text from the ChatGPT API

First, we need a new route so that we can stream text from the ChatGPT API. We will call it /app/openai/stream/route.ts.

In the route below we use the OpenAI SDK to request a completion from the API - and we let the Vercel AI SDK handle the streaming to the client-side.

/app/openai/stream/route.ts

import getOpenAIClient from '@/lib/openai-client';
import { OpenAIStream, StreamingTextResponse } from 'ai';
import { NextRequest } from 'next/server';
// you can change this to any model you want
const MODEL = 'gpt-3.5-turbo' as const;
export async function POST(req: NextRequest) {
  const { prompt } = await req.json();
  const client = getOpenAIClient();
  const response = await client.chat.completions.create({
    model: MODEL,
    stream: true,
    messages: getPromptMessages(prompt),
    max_tokens: 500,
  });
  const stream = OpenAIStream(response);
  return new StreamingTextResponse(stream);
}
function getPromptMessages(topic: string) {
  return [
    {
      content: `Given the topic "${topic}", return a list of possible titles for a blog post. Separate each title with a new line.`,
      role: 'user' as const,
    },
  ];
}

In the prompt, we're specifically asking to separate each title with a new line. This will make it easier to display the titles in the client since we can simply split the text by the new line character and return an array of strings.

This is not the best way to return structured data using the ChatGPT API, but it is the easiest way - and it works well for our use case since it's fairly simple for ChatGPT to do well.

Creating the Form

First, we want to create the form to request the user's input. We will split the form into two steps:

Inputting the Topic - Ask the user the topic they want to write about
Selecting a Title - Let the user select a title from the list of titles generated and streamed by ChatGPT

Introducing the Vercel AI SDK

The Vercel AI SDK is a package that makes it easy to use the Route Handler API and the Streaming Text API. It is available on npm as ai.

Similarly to SWR, we can use hooks to request the API. The hook we will use is called useCompletion:

const {
  complete,
  isLoading,
  completion,
  setCompletion,
  stop
} = useCompletion({ api: '/openai/stream' });

The hook can be called imperatively using the complete function - and can be stopped using the stop function. The isLoading property is a boolean that indicates whether the API is currently loading. The completion property is the text streamed by the API. Finally, the setCompletion function can be used to set the completion.

We will use all of these properties in the next sections.

To transform the text streamed by the API into an array of titles, we will simply split the text by the new line character:

const titles = completion.split('\n');

By doing so, we can iterate over the titles array and display the titles in the client.

Streaming content from ChatGPT using the Vercel AI SDK

First, let's define a component that will stream the content from the ChatGPT API. We will call it StreamTitlesForm.

We will first define the StreamTitlesForm, then we complete the component by adding the CreatePostForm component.

app/(app)/new/CreatePostForm.tsx

'use client';
import { useFormStatus } from 'react-dom';
import { useCompletion } from 'ai/react';
import { useState } from 'react';
import { ArrowLeftIcon, ArrowRightIcon } from 'lucide-react';
import {Label} from "@/components/ui/label";
import {Button} from "@/components/ui/button";
import {Input} from "@/components/ui/input";
function StreamTitlesForm(
  { 
    setTitle 
  }: { 
    setTitle: (title: string) => void; 
  }
) {
  const {
    complete,
    isLoading,
    completion,
    setCompletion,
    stop
  } = useCompletion({ api: '/openai/stream' });
  if (completion) {
    const titles = completion.split('\n');
    return (
      <div className='flex flex-col space-y-4'>
        <div>
          <h2 className='text-lg font-semibold'>Select a title</h2>
          <h3 className='text-gray-400'>
            Click on a title to select it and continue.
          </h3>
        </div>
        <div className='flex flex-col space-y-2'>
          {titles.map((title, index) => {
            return (
              <div
                onClick={() => setTitle(title)}
                role='button'
                key={index}
                className='
                p-4 border rounded-xl text-sm hover:bg-gray-50 
                cursor-pointer dark:hover:bg-slate-800'
              >
                {title}
              </div>
            )
          })}
        </div>
        <div className='flex space-x-2 items-center'>
          <Button
            className='flex items-center space-x-1'
            variant={'ghost'}
            onClick={() => {
              stop();
              setCompletion('')
            }}
          >
            <ArrowLeftIcon className='w-4 h-4' />
            <span>Try a different topic</span>
          </Button>
        </div>
      </div>
    );
  }
  return (
    <form onSubmit={(e) => {
      e.preventDefault();
      const target = e.target as HTMLFormElement;
      const topic = new FormData(target).get('topic') as string;
      // request stream
      complete(topic);
    }}>
      <div className='flex flex-col space-y-4'>
        <div>
          <Label className='flex flex-col space-y-2'>
            <span>Enter the topic of your post</span>
            <Input
              name='topic'
              placeholder='Ex. A post about Next.js'
              required
            />
            <span className='text-xs text-gray-400'>
              Be as specific as possible. For example, instead of &quot;A post 
              about Next.js&quot;, try &quot;A post about Next.js and how to 
              use it with Supabase&quot;.
            </span>
          </Label>
        </div>
        <div>
          <GenerateTitlesButton isLoading={isLoading} />
        </div>
      </div>
    </form>
  );
}
function GenerateTitlesButton({ isLoading }: { isLoading: boolean }) {
  return (
    <Button className="flex space-x-2.5" disabled={isLoading}>
      {isLoading ? (
        'Generating titles...'
      ) : (
        <>
          <span>Generate Titles</span>
          <ArrowRightIcon className="w-4 h-4" />
        </>
      )}
    </Button>
  );
}

Explaining the code above:

We use the useCompletion hook to request the API. We pass the api property to the hook to specify the API we want to request. In our case it's /openai/stream.
The completion property is the text streamed by the API. We use it to display the titles. This is only defined if the user has submitted the form. Hence, we can use it to conditionally render the form or the titles.
The complete function is used to request the API. We call it when the user submits the form.
The stop function is used to stop the API request. We call it when the user wants to try a different topic.
The setCompletion function is used to set the completion. We call it when the user types the topic and submits the form
The isLoading property is a boolean that indicates whether the API is currently loading. We use it to disable the button when the API is loading.
The setTitle function is a callback function used to set the title. We call it when the user clicks on a title.
We create a list of titles by splitting the completion by the new line character \n. We then iterate over the titles and display them in the client. When the user clicks on a title, we call the setTitle function to set the title.

Below is a demo:

Then, we define a component that will display the form and the generated titles. We will call it CreatePostForm.

If the user picked a title, we let them confirm their choice and generate the article
If the user didn't pick a title, we display the form to generate the titles.

The Server Action is not yet implemented. We will implement it in the next sections and will add it to the form.

app/(app)/new/CreatePostForm.tsx

'use client';
function CreatePostForm() {
  const [title, setTitle] = useState('');
  if (title) {
    return (
      <form>
        <input type='hidden' name='title' value={title} />
        <div className='flex flex-col space-y-8'>
          <div className='flex flex-col space-y-2'>
            <div>
              You are creating the following post:
            </div>
            <div className='p-4 border rounded-xl text-sm'>
              {title}
            </div>
          </div>
          <div className='flex space-x-2'>
            <Button
              className='flex space-x-2.5 items-center'
              variant={'ghost'}
              onClick={() => setTitle('')}
            >
              <ArrowLeftIcon className='w-4 h-4' />
              <span>Go Back</span>
            </Button>
            <SubmitButton />
          </div>
        </div>
      </form>
    );
  }
  return <StreamTitlesForm setTitle={setTitle} />
}
function SubmitButton() {
  const { pending } = useFormStatus();
  if (pending) {
    return <Button disabled={true}>Creating article...</Button>;
  }
  return (
    <Button className="flex space-x-2 items-center">
      <span>Create Article</span>
      <ArrowRightIcon className="w-4 h-4" />
    </Button>
  );
}
export default CreatePostForm;

Explaining the code above:

We use the setTitle function to set the title when the user clicks on a title.
We use the title state to conditionally render the form or the title.
If the user picked a title, we display the title and a button to confirm the choice and generate the article. Otherwise, we display the form to generate the titles that we defined above.
We define the SubmitButton component separately so we can use the experimental useFormStatus hook to get the status of the form. We use it to disable the button when the form is pending. We also use it to change the text of the button to Creating article... when the form is pending.

Using the experimental "useFormStatus" hook

The useFormStatus hook is experimental, so it might change in the future.

This hook needs to be a child of the "form" element, and it only works when used within client components, which is why we need to use a separate component for the button.

Adding the form to the New Post page

Now that we have a form, we need to display it on the new page that we will create at /app/(app)/new/page.tsx:

/app/(app)/new/page.tsx

import { PencilIcon } from 'lucide-react';
import CreatePostForm from './CreatePostForm';
function NewPostPage() {
  return (
    <div className='container'>
      <div className='flex flex-col flex-1 space-y-8'>
        <div className='flex justify-between items-start'>
          <h1 className='text-2xl font-semibold flex space-x-4 items-center'>
            <PencilIcon className='w-5 h-5' />
            <span>
              New Post
            </span>
          </h1>
        </div>
        <CreatePostForm />
      </div>
    </div>
  );
}
export default NewPostPage;

Using Server Actions to request the ChatGPT API

To generate our blog post content using the ChatGPT API, we will use a Server Action. As we've seen before, server actions allow us to execute server side code without defining an API route - but instead, using simple functions.

Create the posts Server Actions

To implement the server actions, we create a new file at lib/actions we call posts.ts.

By adding use server at the top of the file, all the exported functions will automatically become server actions.

Now that we have our OpenAI client, we can create a function to generate a blog post. We will call it generatePostAction.

When used using the action property of the form tag, the function accepts a FormData object as an argument.

First, we want to define the parameters that we want to collect from the user. We will collect the following:

interface GeneratePostParams {
  title: string;
}

We also want to define a constant MODEL that we will use to generate the blog post. We will use the gpt-3.5-turbo model, which is the most balanced model available on OpenAI's API given the current pricing. Feel free to change it to any model you want.

const MODEL = `gpt-3.5-turbo`;

Now, we create the function, that given the parameters, will generate the blog post content. We will call it generatePostContent:

import OpenAI from 'openai';
import ChatCompletion = OpenAI.ChatCompletion;
async function generatePostContent(params: GeneratePostParams) {
  const client = getOpenAIClient();
  const content = getCreatePostPrompt(params);
  const response = await client.chat.completions.create({
    temperature: 0.7,
    model: MODEL,
    max_tokens: 500,
    messages: [
      {
        role: 'user' as const,
        content,
      },
    ],
  });
 const usage = response.usage?.total_tokens ?? 0;
 const text = getResponseContent(response);
  return {
    text,
    usage,
  };
}
function getCreatePostPrompt(params: GeneratePostParams) {
  return `
    Write a blog post under 500 words whose title is "${params.title}".
  }
  `;
}
function getResponseContent(response: ChatCompletion) {
  return (response.choices ?? []).reduce((acc, choice) => {
    return acc + (choice.message?.content ?? '');
  }, '');
}

Feel free to edit the parameters of the createChatCompletion function to change the way the blog post is generated. I've added some sensible defaults, but you can change them to your liking.

The amount of tokens set (500) may not be enough to generate a blog post. If your posts are not completed, you can increase the amount of tokens. In the future, we will support automatically requesting more tokens if the post is not completed.

Finally, we export the server action that will be called by the form.

export async function createPostAction(formData: FormData) {
  const title = formData.get('title') as string;
  const { text } = await generatePostContent({ title });
  return {
    text,
  };
}

Here is the full source code of the lib/actions/posts.ts server functions:

lib/actions/posts.ts

"use server";
import OpenAI from 'openai';
import ChatCompletion = OpenAI.ChatCompletion;
import getOpenAIClient from '@/lib/openai-client';
interface GeneratePostParams {
  title: string;
}
const MODEL = `gpt-3.5-turbo`;
export async function createPostAction(formData: FormData) {
  const title = formData.get('title') as string;
  const { text } = await generatePostContent({ title });
  return {
    text,
  };
}
async function generatePostContent(params: GeneratePostParams) {
  const client = getOpenAIClient();
  const content = getCreatePostPrompt(params);
  const response = await client.chat.completions.create({
    temperature: 0.8,
    model: MODEL,
    max_tokens: 500,
    messages: [
      {
        role: 'user',
        content,
      },
    ],
  });
  const usage = response.usage?.total_tokens ?? 0;
  const text = getResponseContent(response);
  return {
    text,
    usage,
  };
}
function getCreatePostPrompt(params: GeneratePostParams) {
  return `
    Write a blog post under 500 words whose title is "${params.title}".
  `;
}
function getResponseContent(response: ChatCompletion) {
  return (response.choices ?? []).reduce((acc, choice) => {
    return acc + (choice.message?.content ?? '');
  }, '');
}

NB: we've added the use server comment at the top of the file to enable server actions. In this way, all the exported functions will automatically become server actions.

About the prompt...

The prompt for generating the post is super simple - I am sure you can come up with a better one!

Adding the action to the form

Now, we will use the server action createPostAction as the action of the form.

Let's go back to CreatePostForm and add the action to the form at line 122:

app/(app)/new/CreatePostForm.tsx

'use client';
import { useState } from "react";
import { useFormStatus } from "react-dom";
import { useCompletion } from "ai/react";
import { ArrowLeftIcon, ArrowRightIcon } from "lucide-react";
import { Label } from "@/components/ui/label";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import { createPostAction } from "@/lib/actions/posts";
function StreamTitlesForm({ setTitle }: { setTitle: (title: string) => void }) {
  const { complete, isLoading, completion, setCompletion, stop } =
    useCompletion({ api: "/openai/stream" });
  if (completion) {
    const titles = completion.split("\n");
    return (
      <div className="flex flex-col space-y-4">
        <div>
          <h2 className="text-lg font-semibold">Select a title</h2>
          <h3 className="text-gray-400">
            Click on a title to select it and continue.
          </h3>
        </div>
        <div className="flex flex-col space-y-2">
          {titles.map((title, index) => {
            return (
              <div
                onClick={() => setTitle(title)}
                role="button"
                key={index}
                className="
                p-4 border rounded-xl text-sm hover:bg-gray-50
                cursor-pointer dark:hover:bg-slate-800"
              >
                {title}
              </div>
            );
          })}
        </div>
        <div className="flex space-x-2 items-center">
          <Button
            className="flex items-center space-x-1"
            variant={"ghost"}
            onClick={() => {
              stop();
              setCompletion("");
            }}
          >
            <ArrowLeftIcon className="w-4 h-4" />
            <span>Try a different topic</span>
          </Button>
        </div>
      </div>
    );
  }
  return (
    <form
      onSubmit={(e) => {
        e.preventDefault();
        const target = e.target as HTMLFormElement;
        const topic = new FormData(target).get("topic") as string;
        // request stream
        complete(topic);
      }}
    >
      <div className="flex flex-col space-y-4">
        <div>
          <Label className="flex flex-col space-y-2">
            <span>Enter the topic of your post</span>
            <Input
              name="topic"
              placeholder="Ex. A post about Next.js"
              required
            />
            <span className="text-xs text-gray-400">
              Be as specific as possible. For example, instead of &quot;A post
              about Next.js&quot;, try &quot;A post about Next.js and how to use
              it with Supabase&quot;.
            </span>
          </Label>
        </div>
        <div>
          <GenerateTitlesButton isLoading={isLoading} />
        </div>
      </div>
    </form>
  );
}
function GenerateTitlesButton({ isLoading }: { isLoading: boolean }) {
  return (
    <Button className="flex space-x-2.5" disabled={isLoading}>
      {isLoading ? (
        "Generating titles..."
      ) : (
        <>
          <span>Generate Titles</span>
          <ArrowRightIcon className="w-4 h-4" />
        </>
      )}
    </Button>
  );
}
function CreatePostForm() {
  const [title, setTitle] = useState("");
  if (title) {
    return (
      <form action={createPostAction}>
        <input type="hidden" name="title" value={title} />
        <div className="flex flex-col space-y-8">
          <div className="flex flex-col space-y-2">
            <div>You are creating the following post:</div>
            <div className="p-4 border rounded-xl text-sm">{title}</div>
          </div>
          <div className="flex space-x-2">
            <Button
              className="flex space-x-2.5 items-center"
              variant={"ghost"}
              onClick={() => setTitle("")}
            >
              <ArrowLeftIcon className="w-4 h-4" />
              <span>Go Back</span>
            </Button>
            <SubmitButton />
          </div>
        </div>
      </form>
    );
  }
  return <StreamTitlesForm setTitle={setTitle} />;
}
function SubmitButton() {
  const { pending } = useFormStatus();
  if (pending) {
    return <Button disabled={true}>Creating article...</Button>;
  }
  return (
    <Button className="flex space-x-2 items-center">
      <span>Create Article</span>
      <ArrowRightIcon className="w-4 h-4" />
    </Button>
  );
}
export default CreatePostForm;

Inserting the generated posts into the database

At the moment, the generated content is simply returned back to the frontend. We now want to insert it into the posts table, and then we navigate to the post page so that the user can see the generated post.

Creating a Mutation with Supabase

We will use the Supabase SDK to insert the generated post into the posts table.

My preferred way to do so is to create functions that accept the Supabase client as a parameter and the parameters of the mutation. This allows us to abstract the client and reuse the functions in both the client and the server.

First, we want to create a function to insert a post into the posts table. We will call it insertPost.

import { Database } from '@/database.types';
import { SupabaseClient } from '@supabase/supabase-js';
type Client = SupabaseClient<Database>;
interface InsertPostParams {
  title: string;
  content: string;
  user_id: string;
}
export async function insertPost(
  client: Client,
  params: InsertPostParams
) {
  const { data, error } = await client
    .from('posts')
    .insert(params)
    .select('uuid')
    .single();
  if (error) {
    throw error;
  }
  return data;
}

In the above, we use the Supabase SDK to insert the post into the posts table.

Additionally, we use the select function to return the uuid of the post from the mutation response, which we will use to redirect the user to the post page.

What is the "single" modifier in Supabase?

The single modifier lets us flatten the response from an array to a single element, so that the response will be { uuid: string } instead of Array<{ uuid: string }>.

Good to know about using single in your Supabase queries - the single modifier will throw an error in two cases:

Empty: The response is an empty array
Multiple elements: The response is an array with more than one element

If you are not sure whether the response will be an array or a single element, you can use the maybeSingle modifier instead. It will return null if the response is an empty array, and will return the first element if the response is an array with more than one element.

 const { data, error } = await client
    .from('posts')
    .insert(params)
    .select('uuid')
    .maybeSingle();

This does not apply to our case since we are inserting a single post, but it's good to know about it.

Updating the Server Action to use the mutation

Now, we want to update the server action and insert the post into the database, and then redirect the user to the post's page - which we haven't implemented yet, but we will in the next section.

Let's go back to lib/actions/posts.ts and update the createPostAction function.

import { redirect } from "next/navigation";
import { revalidatePath } from 'next/cache';
import getOpenAIClient from "@/lib/openai-client";
import getSupabaseServerActionClient from "@/lib/supabase/action-client";
import { insertPost } from "@/lib/mutations/posts";
export async function createPostAction(formData: FormData) {
  const title = formData.get('title') as string;
  const { text: content } = await generatePostContent({
    title,
  });
  // log the content to see the result!
  console.log(content);
  const client = getSupabaseServerActionClient();
  const { data, error } = await client.auth.getUser();
  if (error) {
    throw error;
  }
  const { uuid } = await insertPost(client, {
    title,
    content,
    user_id: data.user.id
  });
  revalidatePath(`/dashboard`, 'page');
  // redirect to the post page.
  // NB: it will return a 404 error since we haven't implemented the post page yet
  return redirect(`/dashboard/${uuid}`);
}

As you can see above, we use the insertPost function to insert the post into the database.
We also use the function revalidatePath to revalidate the /dashboard page. This will ensure that the user will see the new post in the dashboard - since Next.js will cache the page if the user has already visited it.
We then use the redirect function to redirect the user to the post's page. For the time being, it will return a 404 error since we haven't implemented the post page yet.

Reading the generated content

Since we cannot read the content yet, we can test it works by logging the result to the console.

Also, we can check that the post has been inserted into the database by checking the Supabase Studio:

We will implement the post page in the next section.

We will get back to this section later

Once we implement Stripe subscriptions and quotas, we will need to read the tokens used by the user to generate the post. We will then need to update our DB so we can keep track of the tokens used by the user.

This is needed so that the user won't be able to generate posts if they don't have enough tokens left and prevent them from going over their quota.

Conclusion

In this section, we learned how to generate and stream content using the OpenAI's ChatGPT API and the AI SDK from Vercel.
We also learned how to insert content into the "posts" table with the Supabase SDK.
Finally, we learned how to use Next.js Server Actions to perform server requests without defining an API route.

What's Next

In the next section, we will learn how to display the posts on the front-end.

We will be creating a new dynamic route at /dashboard/[uuid] to display the posts, read the parameters from the URL and fetch the post from the database.

Finally, we will also implement some CRUD operations, such as updating and deleting posts.