> **Can't find what you're looking for?** Use `search_docs` on the docs MCP server at `https://viem.sh/api/mcp` to find what you need.

# Run Inference

:::warning
The Gateway API is experimental and may change in future releases.
:::

## Overview

`client.ai` reaches the Gateway's models. Calls are metered per token: each request reserves the most it can
cost, which `maxOutputTokens` bounds, and the unused part is credited back to your payment channel when it
ends. Set `maxOutputTokens` to what you need so a call doesn't hold more of your channel than it uses.

## Recipes

These recipes assume you have [set up a Gateway client](/tempo/guides/gateway/connect).

### Pick a Model

:::code-group
```ts twoslash [example.ts]
import { client } from './viem.config'

const { models, tokens } = await client.gateway.getCapabilities() // [!code focus]
const model = models.generate[0] // models.embed lists embedding models
// tokens[model]: { input, output } in USD per million tokens
```

```ts twoslash [viem.config.ts] filename="viem.config.ts"
// [!include ~/snippets/tempo/gateway.config.ts:setup]
```
:::

### Generate Text

Pass a `prompt`, or `messages` for a conversation.

:::code-group
```ts twoslash [example.ts]
import { client } from './viem.config'

const { text, usage } = await client.ai.generateText({ // [!code focus]
  maxOutputTokens: 256, // [!code focus]
  model: '@cf/zai-org/glm-5.3', // [!code focus]
  prompt: 'Summarize Tempo in one sentence.', // [!code focus]
}) // [!code focus]
```

```ts twoslash [viem.config.ts] filename="viem.config.ts"
// [!include ~/snippets/tempo/gateway.config.ts:setup]
```
:::

:::code-group
```ts twoslash [example.ts]
import { client } from './viem.config'

const { text } = await client.ai.generateText({
  maxOutputTokens: 512,
  messages: [ // [!code focus]
    { content: 'You answer in one short paragraph.', role: 'system' }, // [!code focus]
    { content: 'What is a payment channel?', role: 'user' }, // [!code focus]
  ], // [!code focus]
  model: '@cf/zai-org/glm-5.3',
})
```

```ts twoslash [viem.config.ts] filename="viem.config.ts"
// [!include ~/snippets/tempo/gateway.config.ts:setup]
```
:::

### Stream Text

`ai.streamText` is an async iterator. It yields `text` and `reasoning` deltas as the model writes, then a
final event with usage. An error from the Gateway mid-stream throws `Gateway.RequestError`.

:::code-group
```ts twoslash [example.ts]
import { client } from './viem.config'

for await (const event of client.ai.streamText({ // [!code focus]
  model: '@cf/zai-org/glm-5.3',
  prompt: 'Write a haiku about stablecoins.',
})) {
  if (event.type === 'text') process.stdout.write(event.delta) // [!code focus]
}
```

```ts twoslash [viem.config.ts] filename="viem.config.ts"
// [!include ~/snippets/tempo/gateway.config.ts:setup]
```
:::

Pass a `signal` to stop a stream early, or `break` out of the loop: the request is cancelled, and you pay only
for the tokens generated.

:::code-group
```ts twoslash [example.ts]
import { client } from './viem.config'

const controller = new AbortController() // [!code focus]
setTimeout(() => controller.abort(), 5_000) // [!code focus]

for await (const event of client.ai.streamText({
  model: '@cf/zai-org/glm-5.3',
  prompt: 'Write a long story about stablecoins.',
  signal: controller.signal, // [!code focus]
})) {
  if (event.type === 'text') process.stdout.write(event.delta)
}
```

```ts twoslash [viem.config.ts] filename="viem.config.ts"
// [!include ~/snippets/tempo/gateway.config.ts:setup]
```
:::

### Embed Text

Pick an embedding model from `models.embed`: a Gateway lists only the models it serves, and the list can be empty.

:::code-group
```ts twoslash [example.ts]
import { client } from './viem.config'

const { models } = await client.gateway.getCapabilities()
const [model] = models.embed // [!code focus]
if (!model) throw new Error('This Gateway serves no embedding models.')

const { embeddings } = await client.ai.embed({ // [!code focus]
  input: ['Tempo is a payments chain.', 'Viem is a TypeScript interface for Ethereum.'], // [!code focus]
  model, // [!code focus]
}) // [!code focus]
```

```ts twoslash [viem.config.ts] filename="viem.config.ts"
// [!include ~/snippets/tempo/gateway.config.ts:setup]
```
:::

### Keep a Conversation

Store the history in the Gateway's key-value store so it survives across sessions and devices, in the same
app namespace as the rest of your data.

:::code-group
```ts twoslash [example.ts]
import { client } from './viem.config'

type Message = { content: string; role: 'assistant' | 'system' | 'user' }

async function ask(chatId: string, question: string) {
  const key = `chats/${chatId}`
  const saved = await client.kv.get({ key }) // [!code focus]
  const history = (saved?.value as { messages: Message[] } | undefined)?.messages ?? []
  const messages: Message[] = [...history, { content: question, role: 'user' }]

  const { text } = await client.ai.generateText({
    maxOutputTokens: 512,
    messages,
    model: '@cf/zai-org/glm-5.3',
  })

  await client.kv.set({ // [!code focus]
    ifVersion: saved?.version ?? null, // [!code focus]
    key, // [!code focus]
    value: { messages: [...messages, { content: text, role: 'assistant' }] }, // [!code focus]
  }) // [!code focus]
  return text
}
```

```ts twoslash [viem.config.ts] filename="viem.config.ts"
// [!include ~/snippets/tempo/gateway.config.ts:setup]
```
:::

## See More

<Cards>
  <Card icon="lucide:database" title="Store Objects & State" description="Keep files in a bucket and JSON state in a versioned key-value store." to="/tempo/guides/gateway/storage" />

  <Card icon="lucide:square-function" title="Gateway" description="Reference for every ai method." to="/tempo/utilities/Gateway" />
</Cards>