---
description: Call AI models from Workers AI, OpenAI, Anthropic, Google Gemini, or any provider within Cloudflare Agents.
title: Using AI Models
image: https://developers.cloudflare.com/og-docs.png
---

[Skip to content](#main-content)

> Documentation Index  
> Fetch the complete documentation index at: https://developers.cloudflare.com/agents/llms.txt  
> Use this file to discover all available pages before exploring further.

# Using AI Models

Last updated Aug 24, 2026|Copy as Markdown|[View as Markdown](https://e8aee267.previews.developers.cloudflare.com/agents/runtime/operations/using-ai-models/index.md)|[Agent setup](https://e8aee267.previews.developers.cloudflare.com/agent-setup/)

Agents can call AI models from any provider. [Workers AI](https://e8aee267.previews.developers.cloudflare.com/workers-ai/) is built in and requires no API keys. You can also use [OpenAI ↗](https://platform.openai.com/docs/quickstart?language=javascript), [Anthropic ↗](https://docs.anthropic.com/en/api/client-sdks#typescript), [Google Gemini ↗](https://ai.google.dev/gemini-api/docs/openai), or any service that exposes an OpenAI-compatible API.

The [AI SDK ↗](https://ai-sdk.dev/docs/introduction) provides a unified interface across all of these providers, and is what `AIChatAgent` and the starter template use under the hood. You can also use the model routing features in [AI Gateway](https://e8aee267.previews.developers.cloudflare.com/ai-gateway/) to route across providers, eval responses, and manage rate limits.

## Calling AI Models

You can call models from any method within an Agent, including from HTTP requests using the [onRequest](https://e8aee267.previews.developers.cloudflare.com/agents/runtime/agents-api/) handler, when a [scheduled task](https://e8aee267.previews.developers.cloudflare.com/agents/runtime/execution/schedule-tasks/) runs, when handling a WebSocket message in the [onMessage](https://e8aee267.previews.developers.cloudflare.com/agents/runtime/communication/websockets/) handler, or from any of your own methods.

Agents can call AI models on their own — autonomously — and can handle long-running responses that take minutes (or longer) to respond in full. If a client disconnects mid-stream, the Agent keeps running and can catch the client up when it reconnects.

### Streaming over WebSockets

Modern reasoning models can take some time to both generate a response _and_ stream the response back to the client. Instead of buffering the entire response, you can stream it back over [WebSockets](https://e8aee267.previews.developers.cloudflare.com/agents/runtime/communication/websockets/).

```js
import { Agent } from "agents";
import { streamText } from "ai";
import { createWorkersAI } from "workers-ai-provider";

export class MyAgent extends Agent {
	async onConnect(connection, ctx) {
		//
	}

	async onMessage(connection, message) {
		let msg = JSON.parse(message);
		await this.queryReasoningModel(connection, msg.prompt);
	}

	async queryReasoningModel(connection, userPrompt) {
		try {
			const workersai = createWorkersAI({ binding: this.env.AI });
			const result = streamText({
				model: workersai("@cf/zai-org/glm-4.7-flash"),
				prompt: userPrompt,
			});

			for await (const chunk of result.textStream) {
				if (chunk) {
					connection.send(JSON.stringify({ type: "chunk", content: chunk }));
				}
			}

			connection.send(JSON.stringify({ type: "done" }));
		} catch (error) {
			connection.send(JSON.stringify({ type: "error", error: error }));
		}
	}
}
```

```ts
import { Agent } from "agents";
import { streamText } from "ai";
import { createWorkersAI } from "workers-ai-provider";

interface Env {
	AI: Ai;
}

export class MyAgent extends Agent<Env> {
	async onConnect(connection: Connection, ctx: ConnectionContext) {
		//
	}

	async onMessage(connection: Connection, message: WSMessage) {
		let msg = JSON.parse(message);
		await this.queryReasoningModel(connection, msg.prompt);
	}

	async queryReasoningModel(connection: Connection, userPrompt: string) {
		try {
			const workersai = createWorkersAI({ binding: this.env.AI });
			const result = streamText({
				model: workersai("@cf/zai-org/glm-4.7-flash"),
				prompt: userPrompt,
			});

			for await (const chunk of result.textStream) {
				if (chunk) {
					connection.send(JSON.stringify({ type: "chunk", content: chunk }));
				}
			}

			connection.send(JSON.stringify({ type: "done" }));
		} catch (error) {
			connection.send(JSON.stringify({ type: "error", error: error }));
		}
	}
}
```

You can also persist AI model responses back to [Agent state](https://e8aee267.previews.developers.cloudflare.com/agents/runtime/lifecycle/state/) using `this.setState`. If a user disconnects, read the message history back and send it to the user when they reconnect.

## Workers AI

You can use [any of the models available in Workers AI](https://e8aee267.previews.developers.cloudflare.com/workers-ai/models/) within your Agent by [configuring a binding](https://e8aee267.previews.developers.cloudflare.com/workers-ai/configuration/bindings/). No API keys are required.

Workers AI supports streaming responses by setting `stream: true`. Use streaming to avoid buffering and delaying responses, especially for larger models or reasoning models.

```js
import { Agent } from "agents";

export class MyAgent extends Agent {
	async onRequest(request) {
		const stream = await this.env.AI.run(
			"@cf/deepseek-ai/deepseek-r1-distill-qwen-32b",
			{
				prompt: "Build me a Cloudflare Worker that returns JSON.",
				stream: true,
			},
		);

		return new Response(stream, {
			headers: { "content-type": "text/event-stream" },
		});
	}
}
```

```ts
import { Agent } from "agents";

interface Env {
	AI: Ai;
}

export class MyAgent extends Agent<Env> {
	async onRequest(request: Request) {
		const stream = await this.env.AI.run(
			"@cf/deepseek-ai/deepseek-r1-distill-qwen-32b",
			{
				prompt: "Build me a Cloudflare Worker that returns JSON.",
				stream: true,
			},
		);

		return new Response(stream, {
			headers: { "content-type": "text/event-stream" },
		});
	}
}
```

Your Wrangler configuration needs an `ai` binding:

```jsonc
{
	"ai": {
		"binding": "AI",
	},
}
```

```toml
[ai]
binding = "AI"
```

### Model routing

You can use [AI Gateway](https://e8aee267.previews.developers.cloudflare.com/ai-gateway/) directly from an Agent by specifying a [gateway configuration](https://e8aee267.previews.developers.cloudflare.com/ai-gateway/usage/providers/workersai/) when calling the AI binding. Model routing lets you route requests across providers based on availability, rate limits, or cost budgets.

```js
import { Agent } from "agents";

export class MyAgent extends Agent {
	async onRequest(request) {
		const response = await this.env.AI.run(
			"@cf/deepseek-ai/deepseek-r1-distill-qwen-32b",
			{
				prompt: "Build me a Cloudflare Worker that returns JSON.",
			},
			{
				gateway: {
					id: "{gateway_id}",
					skipCache: false,
					cacheTtl: 3360,
				},
			},
		);

		return Response.json(response);
	}
}
```

```ts
import { Agent } from "agents";

interface Env {
	AI: Ai;
}

export class MyAgent extends Agent<Env> {
	async onRequest(request: Request) {
		const response = await this.env.AI.run(
			"@cf/deepseek-ai/deepseek-r1-distill-qwen-32b",
			{
				prompt: "Build me a Cloudflare Worker that returns JSON.",
			},
			{
				gateway: {
					id: "{gateway_id}",
					skipCache: false,
					cacheTtl: 3360,
				},
			},
		);

		return Response.json(response);
	}
}
```

The `ai` binding in your Wrangler configuration is shared across both Workers AI and AI Gateway.

```jsonc
{
	"ai": {
		"binding": "AI",
	},
}
```

```toml
[ai]
binding = "AI"
```

Visit the [AI Gateway documentation](https://e8aee267.previews.developers.cloudflare.com/ai-gateway/) to learn how to configure a gateway and retrieve a gateway ID.

## AI SDK

The [AI SDK ↗](https://ai-sdk.dev/docs/introduction) provides a unified API for text generation, tool calling, structured responses, and more. It works with any provider that has an AI SDK adapter, including Workers AI via [workers-ai-provider ↗](https://www.npmjs.com/package/workers-ai-provider).

npmyarnpnpmbun

```
npm i ai workers-ai-provider
```

```
yarn add ai workers-ai-provider
```

```
pnpm add ai workers-ai-provider
```

```
bun add ai workers-ai-provider
```

```js
import { Agent } from "agents";
import { generateText } from "ai";
import { createWorkersAI } from "workers-ai-provider";

export class MyAgent extends Agent {
	async onRequest(request) {
		const workersai = createWorkersAI({ binding: this.env.AI });
		const { text } = await generateText({
			model: workersai("@cf/zai-org/glm-4.7-flash"),
			prompt: "Build me an AI agent on Cloudflare Workers",
		});

		return Response.json({ modelResponse: text });
	}
}
```

```ts
import { Agent } from "agents";
import { generateText } from "ai";
import { createWorkersAI } from "workers-ai-provider";

interface Env {
	AI: Ai;
}

export class MyAgent extends Agent<Env> {
	async onRequest(request: Request): Promise<Response> {
		const workersai = createWorkersAI({ binding: this.env.AI });
		const { text } = await generateText({
			model: workersai("@cf/zai-org/glm-4.7-flash"),
			prompt: "Build me an AI agent on Cloudflare Workers",
		});

		return Response.json({ modelResponse: text });
	}
}
```

You can swap the provider to use OpenAI, Anthropic, or any other AI SDK-compatible adapter:

npmyarnpnpmbun

```
npm i ai @ai-sdk/openai
```

```
yarn add ai @ai-sdk/openai
```

```
pnpm add ai @ai-sdk/openai
```

```
bun add ai @ai-sdk/openai
```

```js
import { Agent } from "agents";
import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";

export class MyAgent extends Agent {
	async onRequest(request) {
		const { text } = await generateText({
			model: openai("gpt-4o"),
			prompt: "Build me an AI agent on Cloudflare Workers",
		});

		return Response.json({ modelResponse: text });
	}
}
```

```ts
import { Agent } from "agents";
import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";

export class MyAgent extends Agent {
	async onRequest(request: Request): Promise<Response> {
		const { text } = await generateText({
			model: openai("gpt-4o"),
			prompt: "Build me an AI agent on Cloudflare Workers",
		});

		return Response.json({ modelResponse: text });
	}
}
```

## OpenAI-compatible endpoints

Agents can call models across any service that supports the OpenAI API. For example, you can use the OpenAI SDK to call one of [Google's Gemini models ↗](https://ai.google.dev/gemini-api/docs/openai#node.js) directly from your Agent.

Agents can stream responses back over HTTP using Server-Sent Events (SSE) from within an `onRequest` handler, or by using the native [WebSocket API](https://e8aee267.previews.developers.cloudflare.com/agents/runtime/communication/websockets/) to stream responses back to a client.

```js
import { Agent } from "agents";
import { OpenAI } from "openai";

export class MyAgent extends Agent {
	async onRequest(request) {
		const client = new OpenAI({
			apiKey: this.env.GEMINI_API_KEY,
			baseURL: "https://generativelanguage.googleapis.com/v1beta/openai/",
		});

		let { readable, writable } = new TransformStream();
		let writer = writable.getWriter();
		const textEncoder = new TextEncoder();

		this.ctx.waitUntil(
			(async () => {
				const stream = await client.chat.completions.create({
					model: "gemini-2.0-flash",
					messages: [
						{ role: "user", content: "Write me a Cloudflare Worker." },
					],
					stream: true,
				});

				for await (const part of stream) {
					writer.write(
						textEncoder.encode(part.choices[0]?.delta?.content || ""),
					);
				}
				writer.close();
			})(),
		);

		return new Response(readable);
	}
}
```

```ts
import { Agent } from "agents";
import { OpenAI } from "openai";

export class MyAgent extends Agent {
	async onRequest(request: Request): Promise<Response> {
		const client = new OpenAI({
			apiKey: this.env.GEMINI_API_KEY,
			baseURL: "https://generativelanguage.googleapis.com/v1beta/openai/",
		});

		let { readable, writable } = new TransformStream();
		let writer = writable.getWriter();
		const textEncoder = new TextEncoder();

		this.ctx.waitUntil(
			(async () => {
				const stream = await client.chat.completions.create({
					model: "gemini-2.0-flash",
					messages: [
						{ role: "user", content: "Write me a Cloudflare Worker." },
					],
					stream: true,
				});

				for await (const part of stream) {
					writer.write(
						textEncoder.encode(part.choices[0]?.delta?.content || ""),
					);
				}
				writer.close();
			})(),
		);

		return new Response(readable);
	}
}
```

Was this helpful?

YesNo

## On this page

[![](https://e8aee267.previews.developers.cloudflare.com/_astro/logo.te5VL_aD.svg)Docs](https://e8aee267.previews.developers.cloudflare.com/)

```json
{"@context":"https://schema.org","@type":"TechArticle","@id":"https://developers.cloudflare.com/agents/runtime/operations/using-ai-models/#page","headline":"Using AI Models · Cloudflare Agents docs","description":"Call AI models from Workers AI, OpenAI, Anthropic, Google Gemini, or any provider within Cloudflare Agents.","url":"https://developers.cloudflare.com/agents/runtime/operations/using-ai-models/","inLanguage":"en","image":"https://developers.cloudflare.com/og-docs.png","dateModified":"2026-08-24","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://developers.cloudflare.com/#website","name":"Cloudflare Docs","url":"https://developers.cloudflare.com/"},"keywords":["AI"]}
```