Skip to content
Cloudflare Docs
DirectoryAPISDKsChangelog

Bring your own generation model

By default, AI Search uses a Workers AI model to generate responses. To use a model outside of Workers AI, use AI Search for search and pass the retrieved content to a different model for generation. This guide uses an OpenAI model.

Prerequisites

  1. Sign up for a Cloudflare account ↗.
  2. Install Node.js ↗.

Node.js version manager

Use a Node version manager like Volta ↗ or nvm ↗ to avoid permission issues and change Node.js versions. Wrangler, discussed later in this guide, requires a Node version of 16.17.0 or later.

You also need:

1. Create a Worker project

Create a new Worker project using the create-cloudflare CLI (C3). C3 ↗ is a command-line tool designed to help you set up and deploy new applications to Cloudflare.

Create a new project named byo-model by running:

npm create cloudflare@latest -- byo-model

For setup, select the following options:

  • For What would you like to start with?, choose Hello World example.
  • For Which template would you like to use?, choose Worker only.
  • For Which language do you want to use?, choose TypeScript.
  • For Do you want to use git for version control?, choose Yes.
  • For Do you want to deploy your application?, choose No (we will be making some changes before deploying).

Go to your application directory:

Terminal window
cd byo-model

2. Install the AI SDK and OpenAI provider

Install the AI SDK ↗ and its OpenAI provider:

npm i ai @ai-sdk/openai

3. Bind your Worker and set your API key

Add the AI Search binding to your Wrangler configuration file:

JSONC
{
  "$schema": "./node_modules/wrangler/config-schema.json",
  "ai_search_namespaces": [
    {
      "binding": "AI_SEARCH",
      "namespace": "default",
      "remote": true
    }
  ]
}

Store your OpenAI API key as a secret:

Terminal window
npx wrangler secret put OPENAI_API_KEY

For local development, add the key to a .dev.vars file in your project root instead:

.dev.vars
OPENAI_API_KEY="<YOUR_OPENAI_API_KEY>"

4. Add the code

Update src/index.ts. This Worker searches your instance, formats the retrieved chunks, and passes them to OpenAI to generate an answer. Replace my-instance with the name of your instance.

src/index.js
import { createOpenAI } from "@ai-sdk/openai";
import { generateText } from "ai";


export default {
  async fetch(request, env) {
    const url = new URL(request.url);
    const userQuery = url.searchParams.get("query") ?? "What is Cloudflare?";


    // Search for documents in AI Search.
    const searchResult = await env.AI_SEARCH.get("my-instance").search({
      messages: [{ role: "user", content: userQuery }],
    });


    if (searchResult.chunks.length === 0) {
      return Response.json({ text: `No data found for query "${userQuery}"` });
    }


    // Join the retrieved chunks into a single string.
    const chunks = searchResult.chunks
      .map((chunk) => `<file name="${chunk.item.key}">${chunk.text}</file>`)
      .join("\n\n");


    // Send the query and retrieved content to OpenAI for the answer.
    const openai = createOpenAI({ apiKey: env.OPENAI_API_KEY });
    const generateResult = await generateText({
      model: openai("gpt-4o-mini"),
      messages: [
        {
          role: "system",
          content:
            "You are a helpful assistant. Answer the user question using the provided files.",
        },
        { role: "user", content: chunks },
        { role: "user", content: userQuery },
      ],
    });


    return Response.json({ text: generateResult.text });
  },
};

5. Run and deploy

Start a local development server, then query it at /?query=your+search+terms:

Terminal window
npx wrangler dev

Log in with your Cloudflare account, then deploy your Worker to make it accessible on the Internet:

Terminal window
npx wrangler login
npx wrangler deploy

Next steps

Models Use third-party models natively through AI Gateway.
Search Workers binding Full reference for searching and chatting from a Worker.