AI

#server/ai runs model calls from your server code: text, images, speech, transcription, embeddings and video. The kit ships one provider, Workers AI through the AI binding, and calls no model itself. You work here when you build an AI feature or point a role at a different model.

How it works

1. ai.roles in starter.config.ts(Licensees only) lists each thing your app asks a model for: its modality, and the model each provider uses for it.
2. You call the entry point for that modality with a role, such as runText(event, 'copy', …). A role of another modality fails type checking.
3. server/ai/index.ts(Licensees only) tries the providers in ai.providers order. It skips a provider that names no model for the role, has no method for the modality, or has no binding or key. A failure the provider marks retryable is sent once more, then the next provider gets the same request.
4. The entry point returns the result. It throws a 501 when no provider can serve the role and a 502 when every provider refused.

The shipped roles

Rename them, add more roles of one modality, or point them at other models. A role is an entry such as copy: { modality: 'text', models: { workersai: '@cf/meta/llama-3.1-8b-instruct-fast' } }. Role ids must match /^[a-z][a-z0-9]*$/, because each one becomes part of an environment variable. The build also refuses a role that names a model for a provider that can't serve its modality, and a role no bundled provider serves.

Call a model

Rate-limit anything that spends money per call (Rate limiting):

import { runText } from '#server/ai'

export default defineEventHandler(async (event) => {
  const session = await requireVerifiedSession(event)

  const { title } = await readValidatedBody(
    event,
    z.object({ title: z.string().min(1).max(200) }).parse,
  )

  await requireRateLimit(event, {
    scope: 'ai-copy',
    key: String(session.user.id),
    policy: { max: 30, windowMs: 60 * 60_000, lockoutMs: 15 * 60_000 },
  })

  const blurb = await runText(event, 'copy', {
    system: 'Write two plain sentences for a product listing.',
    messages: [{ role: 'user', content: title }],
    maxTokens: 300,
  })

  return { blurb }
})

A text request takes system, messages and maxTokens. There is no temperature and no streaming. maxTokens caps the whole answer, including thinking on models that think. Workers AI text models can't read an image part in a message, so the adapter refuses such a request.

The other entry points take their request shapes from server/ai/types.ts(Licensees only):

const { data, mediaType } = await runImage(event, 'illustration', {
  prompt: 'A lighthouse at dusk, flat colours',
})

const vectors = await runEmbedding(event, 'search', { input: [title, blurb] })

runImage reads the media type from the bytes (JPEG, PNG or WebP), so store it as the Content-Type when you put the image in R2 (Storage). runEmbedding always takes a list and returns vectors in input order. Where they go is up to you: there is no vector store.

Check before you offer a feature

isAiConfigured(event, role) says whether any bundled provider can serve the role on this deploy, with its binding or key in place. Return it from a route and hide the button rather than show one that answers 501.

Handle a failure yourself

attemptAi(event, role, request) runs the same pass without throwing. It returns the success object or { ok: false, reason, retryable }. The reason is the provider's own message and can quote your prompt back, so log it, but never show it or store it on a row a user sees. The entry points log it as [ai] every provider refused "<role>": … before their 502.

Generate video in a job

runVideo waits minutes for the result, so call it from a background job (Background jobs), not from a request. The route records a row and enqueues; the job settles the row. server/jobs/render-clip.ts:

import { attemptAi } from '#server/ai'
import { defineJob } from '#server/utils/jobs'

export interface Payload {
  clipId: number
  prompt: string
}

export default defineJob<Payload>({
  async run({ clipId, prompt }, event) {
    const result = await attemptAi(event, 'motion', {
      prompt,
      durationSeconds: 5,
      aspectRatio: '16:9',
    })

    if (!result.ok && result.retryable) throw new Error('motion: retry later')

    await useDrizzle(event)
      .update(tables.clips)
      .set(
        result.ok ? { status: 'ready', url: result.url } : { status: 'failed' },
      )
      .where(eq(tables.clips.id, clipId))
  },
})

clips is your own table. A job retries on a throw and every retry is billed again, so throw only when a repeat can help. The URL belongs to the provider and expires; copy the file into R2 if it has to last. For image-to-video, pass imageUrl: a URL the provider can fetch, or a data: URI of the frame.

Change a role's model

Edit the model id in ai.roles, or override it per deploy without a rebuild:

NUXT_AI_WORKERSAI_MODELS_COPY=anthropic/claude-sonnet-5

The variable is NUXT_AI_<PROVIDER>_MODELS_<ROLE> and wins over the config. @cf/… ids are Cloudflare's own models, billed as Workers AI usage. author/model ids are third-party models Cloudflare resells through AI Gateway and bills to your Cloudflare account, so load credits under AI Gateway > Credits first. They also need ai.gateway set; the shipped 'default' works. The model catalogue is at https://developers.cloudflare.com/ai/models/.

Add a provider

Create server/ai/adapters/custom/<id>.ts with a default-exported AiAdapter whose id is the file name, and implement a method per modality it serves. A method must not throw: return { ok: false, reason, retryable }, with retryable only for failures a repeat can fix, such as a 429 or a 5xx. server/ai/adapters/custom/openai.ts:

import type { AiAdapter } from '#server/ai/types'

interface ChatCompletion {
  choices?: { message?: { content?: string | null } }[]
}

const openai: AiAdapter = {
  id: 'openai',
  transport: 'fetch',

  async text(request, config) {
    if (request.messages.some((m) => typeof m.content !== 'string')) {
      return { ok: false, reason: 'text messages only', retryable: false }
    }

    try {
      const response = await $fetch.raw<ChatCompletion>(
        'https://api.openai.com/v1/chat/completions',
        {
          method: 'POST',
          headers: { authorization: `Bearer ${config.key}` },
          body: {
            model: config.model,
            messages: [
              ...(request.system
                ? [{ role: 'system', content: request.system }]
                : []),
              ...request.messages,
            ],
          },
          ignoreResponseError: true,
        },
      )

      if (!response.ok) {
        return {
          ok: false,
          reason: `openai answered ${response.status}`,
          retryable: response.status === 429 || response.status >= 500,
        }
      }

      const text = response._data?.choices?.[0]?.message?.content

      return text
        ? { ok: true, text, provider: 'openai' }
        : { ok: false, reason: 'openai returned no text', retryable: false }
    } catch (error) {
      return { ok: false, reason: `openai: ${String(error)}`, retryable: true }
    }
  },
}

export default openai

Then list it, declare what it serves, and give each role it should serve a model:

ai: {
  enabled: true,
  providers: ['workersai', 'openai'],
  gateway: 'default',
  adapters: {
    openai: { modalities: ['text'], credentials: ['apiKey'] },
  },
  roles: {
    copy: {
      modality: 'text',
      models: {
        workersai: '@cf/meta/llama-3.1-8b-instruct-fast',
        openai: 'gpt-5-mini',
      },
    },
  },
},

The build refuses a provider you added until modalities lists what it serves. The key arrives as config.key from NUXT_AI_OPENAI_API_KEY, and the Worker logs that variable at boot while it is unset. Without a key the provider is skipped. Restart pnpm dev after adding the file. A file named workersai.ts in custom/ replaces the shipped adapter.

Turn AI off

Set ai.enabled: false. No adapter is bundled, and every entry point answers with a 501. You can then drop the ai block from wrangler.jsonc and run pnpm cf-typegen.

Configuration

Good to know

• pnpm dev runs without the AI binding, so every call answers 501 until you run pnpm exec wrangler login and set NUXT_DEV_REMOTE_AI=true in .env. Those calls are real and billed.
• Workers AI checks each model's input against that model's schema and refuses unknown fields with error 5006. The adapter in server/ai/adapters/workersai.ts(Licensees only) maps requests for Flux, the diffusion image models, Aura, MeloTTS and Seedance-style video. A model from another family may need its own mapping there.
• A provider you add receives only the credential named apiKey, as config.key. Other keys you list are read into runtime config but never reach the adapter.
• runTranscription treats silence as a failure and throws a 502. Use attemptAi where empty audio is expected.
• test/unit/ai-registry.test.ts(Licensees only) expects the shipped catalogue to cover all six modalities. If you remove the last role of a modality, update that test with it.