Prev

Remix'ing SSE

Consuming an Event Stream in Remix

Open AI Chat Completion Event Stream

One limitation of a EventSource is that it only supports a GET request.

  • We can't leverage the Remix action we previously used.
  • Instead, we create a Resource route to respond to a GET.
  • We can't submit the prompt as POST formData.
  • Instead, we use a query string parameter.

Remix SSE Handler

The parsed event.data looks like the following, where we're interested in choices[0].delta.content.

event-data.json

{
"id": "chatcmpl-7CHegq3EwN9wDkGv9NO33t2Cw1NER",
"object": "chat.completion.chunk",
"created": 1683162158,
"model": "gpt-3.5-turbo-0301",
"choices": [
{
"delta": {
"content": " remix"
},
"index": 0,
"finish_reason": null
}
]
}

Create a Resource Route

The clientside EventSource requires a GET.

  • We use a Remix Resource Route to convert OpenAI's POST to GET.
  • A Resource Route is really just a route file that doesn't have a default export (a component).

Use a Remix Resource Route to expose OpenAI's POST as a GET

routes/api.chatgpt.ts

export const loader = async ({ request }) => {
let urlSearchParams = new URL(request?.url).searchParams;
let prompt = urlSearchParams.get("prompt");
if (!prompt) {
return new Response("prompt is required", { status: 400 });
}
return fetch("https://api.openai.com/v1/chat/completions", {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
},
body: JSON.stringify({
model: "gpt-3.5-turbo",
stream: true,
messages: [
{
role: "user",
content: prompt,
},
],
}),
});
};

Handle the Event Stream

First we'll setup some boiler plate code.

routes/stream-demo.tsx

function handleChatGPTStream(prompt, onData) {
const eventSource = new EventSource("/api/chatgpt?prompt=" + prompt);
eventSource.onmessage = (event) => {
console.log(event.data);
};
eventSource.onerror = (event) => {
eventSource.close();
};
}
export default function Demo() {
return (
<form>
<label>
Prompt
<textarea required name="prompt" />
</label>
<div>
<button type="submit">Send</button>
</div>
</form>
);
}

How do we know when it's done?

When ChatGPT is finished responding, it will return with a message of [DONE]

routes/stream-demo.tsx

function handleChatGPTStream(prompt, onData) {
const eventSource = new EventSource("/api/chatgpt?prompt=" + prompt);
eventSource.onmessage = (event) => {
if (event?.data === "[DONE]") {
eventSource.close();
} else {
// TODO: Parse event.data
}
};
eventSource.onerror = (event) => {
eventSource.close();
};
}
export default function Demo() {
return (
<form>
<label>
Prompt
<textarea required name="prompt" />
</label>
<div>
<button type="submit">Send</button>
</div>
</form>
);
}

Pull out the additional text from event.data.choices[0].delta.content

routes/stream-demo.tsx

function handleChatGPTStream(prompt, onData) {
const eventSource = new EventSource("/api/chatgpt?prompt=" + prompt);
eventSource.onmessage = (event) => {
if (event?.data === "[DONE]") {
eventSource.close();
} else {
let data = JSON.parse(event.data);
let content = data?.choices?.[0]?.delta?.content;
if (content) {
// Invoke the callback with the new text
onData(content);
}
}
};
eventSource.onerror = (event) => {
eventSource.close();
};
}
export default function Demo() {
return (
<form>
<label>
Prompt
<textarea required name="prompt" />
</label>
<div>
<button type="submit">Send</button>
</div>
</form>
);
}

routes/stream-demo.tsx

function handleChatGPTStream(prompt, onData) {
const eventSource = new EventSource("/api/chatgpt?prompt=" + prompt);
eventSource.onmessage = (event) => {
if (event?.data === "[DONE]") {
eventSource.close();
} else {
let data = JSON.parse(event.data);
let content = data?.choices?.[0]?.delta?.content;
if (content) {
// Invoke the callback with the new text
onData(content);
}
}
};
eventSource.onerror = (event) => {
eventSource.close();
};
}
export default function Demo() {
return (
<form>
<label>
Prompt
<textarea required name="prompt" />
</label>
<div>
<button type="submit">Send</button>
</div>
</form>
);
}

Start with a basic form

  • Previously we had used fetcher.Form but that won't work here because we are trying to consume an EventStream.
  • Add an onSubmit handler that calls handleChatGPTStream
routes/stream-demo.tsx

function handleChatGPTStream(prompt, onData) {
const eventSource = new EventSource("/api/chatgpt?prompt=" + prompt);
eventSource.onmessage = (event) => {
if (event?.data === "[DONE]") {
eventSource.close();
} else {
let data = JSON.parse(event.data);
let content = data?.choices?.[0]?.delta?.content;
if (content) {
// Invoke the callback with the new text
onData(content);
}
}
};
eventSource.onerror = (event) => {
eventSource.close();
};
}
export default function Demo() {
return (
<form
onSubmit={(event) => {
event.preventDefault();
const formData = new FormData(event.target);
handleChatGPTStream(formData.get("prompt"), (newText) => {
// TODO
});
}}
>
<label>
Prompt
<textarea required name="prompt" />
</label>
<div>
<button type="submit">Send</button>
</div>
</form>
);
}

Add some state to track the streamed response text

routes/stream-demo.tsx

function handleChatGPTStream(prompt, onData) {
const eventSource = new EventSource("/api/chatgpt?prompt=" + prompt);
eventSource.onmessage = (event) => {
if (event?.data === "[DONE]") {
eventSource.close();
} else {
let data = JSON.parse(event.data);
let content = data?.choices?.[0]?.delta?.content;
if (content) {
// Invoke the callback with the new text
onData(content);
}
}
};
eventSource.onerror = (event) => {
eventSource.close();
};
}
export default function Demo() {
const [streamedText, setStreamedText] = useState("");
return (
<form
onSubmit={(event) => {
event.preventDefault();
const formData = new FormData(event.target);
handleChatGPTStream(formData.get("prompt"), (newText) => {
setStreamedText((prevText) => prevText + newText);
});
}}
>
<label>
Prompt
<textarea required name="prompt" />
</label>
<div>
<button type="submit">Send</button>
</div>
{streamedText && <pre>{streamedText}</pre>}
</form>
);
}

First we'll setup some boiler plate code.

How do we know when it's done?

When ChatGPT is finished responding, it will return with a message of [DONE]

Pull out the additional text from event.data.choices[0].delta.content

Start with a basic form

  • Previously we had used fetcher.Form but that won't work here because we are trying to consume an EventStream.
  • Add an onSubmit handler that calls handleChatGPTStream

Add some state to track the streamed response text

routes/stream-demo.tsx

function handleChatGPTStream(prompt, onData) {
const eventSource = new EventSource("/api/chatgpt?prompt=" + prompt);
eventSource.onmessage = (event) => {
console.log(event.data);
};
eventSource.onerror = (event) => {
eventSource.close();
};
}
export default function Demo() {
return (
<form>
<label>
Prompt
<textarea required name="prompt" />
</label>
<div>
<button type="submit">Send</button>
</div>
</form>
);
}

Demo