📄 ai-sdk/cookbook/rsc/stream-text-with-chat-prompt

Chat completion can sometimes take a long time to finish, especially when the response is big. In such cases, it is useful to stream the chat completion to the client in real-time. This allows the client to display the new message as it is being generated by the model, rather than have users wait for it to finish.

http://localhost:3000

User: How is it going?

Assistant: All good, how may I help you?

Why is the sky blue?

Send Message

Client

Let's create a simple conversation between a user and a model, and place a button that will call continueConversation.

app/page.tsx

'use client';
import { useState } from 'react';import { Message, continueConversation } from './actions';import { readStreamableValue } from '@ai-sdk/rsc';
// Allow streaming responses up to 30 secondsexport const maxDuration = 30;
export default function Home() {  const [conversation, setConversation] = useState<Message[]>([]);  const [input, setInput] = useState<string>('');
  return (    <div>      <div>        {conversation.map((message, index) => (          <div key={index}>            {message.role}: {message.content}          </div>        ))}      </div>
      <div>        <input          type="text"          value={input}          onChange={event => {            setInput(event.target.value);          }}        />        <button          onClick={async () => {            const { messages, newMessage } = await continueConversation([              ...conversation,              { role: 'user', content: input },            ]);
            let textContent = '';
            for await (const delta of readStreamableValue(newMessage)) {              textContent = `${textContent}${delta}`;
              setConversation([                ...messages,                { role: 'assistant', content: textContent },              ]);            }          }}        >          Send Message        </button>      </div>    </div>  );}

Server

Now, let's implement the continueConversation function that will insert the user's message into the conversation and stream back the new message.

app/actions.ts

'use server';
import { streamText } from 'ai';import { openai } from '@ai-sdk/openai';import { createStreamableValue } from '@ai-sdk/rsc';
export interface Message {  role: 'user' | 'assistant';  content: string;}
export async function continueConversation(history: Message[]) {  'use server';
  const stream = createStreamableValue();
  (async () => {    const { textStream } = streamText({      model: openai('gpt-3.5-turbo'),      system:        "You are a dude that doesn't drop character until the DVD commentary.",      messages: history,    });
    for await (const text of textStream) {      stream.update(text);    }
    stream.done();  })();
  return {    messages: history,    newMessage: stream.value,  };}

On this page

Stream Text with Chat Prompt

Client

Server

Deploy and Scale AI Apps with Vercel.

Vercel delivers the infrastructure and developer experience you need to ship reliable AI-powered applications at scale.

Trusted by industry leaders:

OpenAI
Photoroom

Talk to an expert