> ## Documentation Index
> Fetch the complete documentation index at: https://mcpjam-mintlify-docs-update-pr-3444-1785050029015.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Providers

> API reference for LLM provider configuration

The SDK supports 10 built-in LLM providers and allows custom provider configuration for any OpenAI or Anthropic-compatible endpoint. It can also run models on MCPJam's own credits, so an eval in CI needs no provider key at all.

## Model String Format

All models use the format `provider/model-id`:

```typescript theme={"theme":"css-variables"}
"anthropic/claude-sonnet-4-20250514"
"openai/gpt-4o"
"google/gemini-1.5-pro"
```

***

## MCPJam-hosted (no provider key)

Prefix a model with `mcpjam/` and MCPJam calls the provider for you, billing your organization's credits. The only secret you need is an MCPJam API key — which is usually already in your CI for saving results.

| Property | Value |
| - | - |
| Prefix | `mcpjam/` |
| Environment Variable | `MCPJAM_API_KEY` (an `sk_…` key from Settings → API keys) |

```typescript theme={"theme":"css-variables"}
const agent = new HostRunner({
  tools,
  // The vendor model id follows the prefix.
  model: "mcpjam/anthropic/claude-sonnet-4.5",
  apiKey: process.env.MCPJAM_API_KEY!,
});
```

`apiKey` falls back to `MCPJAM_API_KEY`, so in CI you can leave it out entirely.

#### Available models

`mcpjam/anthropic/claude-*` and `mcpjam/openai/gpt-5*`. Other vendors are not hosted — use their own prefix and key.

#### Which project pays

By default the API key's organization Default project, which is also where results are saved — so nothing extra to configure. Set `MCPJAM_PROJECT_ID` to bill a specific project.

#### Cost

Spend appears under the organization's usage. Eval runs also show an estimated cost per iteration, priced from the tokens used at MCPJam's rate for the model. It can differ slightly from the credits billed.

#### Limits

Handled for you, but worth knowing about if you see one in a log:

* Leases last 30 minutes and are renewed automatically.
* Each lease has its own spend and call ceiling; the SDK mints a fresh one when it is used up.
* Your organization's credit balance and spend budget are checked on every call. "Spending limit reached" means add credits — it is not retried.
* A suite revokes its leases when it finishes. If you build runs by hand, call `releaseMcpjamModelLeases()` in an `afterAll`.

```typescript theme={"theme":"css-variables"}
import { releaseMcpjamModelLeases } from "@mcpjam/sdk";

afterAll(async () => {
  await releaseMcpjamModelLeases();
});
```

***

## Built-in Providers

### Anthropic

| Property | Value |
| - | - |
| Prefix | `anthropic/` |
| Environment Variable | `ANTHROPIC_API_KEY` |

#### Models

| Model ID | Description |
| - | - |
| `claude-sonnet-4-20250514` | Latest Sonnet (recommended) |
| `claude-3-5-sonnet-20241022` | Claude 3.5 Sonnet |
| `claude-3-haiku-20240307` | Fast, efficient |
| `claude-3-opus-20240229` | Most capable |

#### Example

```typescript theme={"theme":"css-variables"}
const agent = new HostRunner({
  tools,
  model: "anthropic/claude-sonnet-4-20250514",
  apiKey: process.env.ANTHROPIC_API_KEY,
});
```

***

### OpenAI

| Property | Value |
| - | - |
| Prefix | `openai/` |
| Environment Variable | `OPENAI_API_KEY` |

#### Models

| Model ID | Description |
| - | - |
| `gpt-4o` | Latest GPT-4 Omni (recommended) |
| `gpt-4o-mini` | Fast, cost-effective |
| `gpt-4-turbo` | GPT-4 Turbo |
| `gpt-4` | Original GPT-4 |
| `gpt-3.5-turbo` | Fast, economical |

#### Example

```typescript theme={"theme":"css-variables"}
const agent = new HostRunner({
  tools,
  model: "openai/gpt-4o",
  apiKey: process.env.OPENAI_API_KEY,
});
```

***

### Azure OpenAI

| Property | Value |
| - | - |
| Prefix | `azure/` |
| Environment Variables | `AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT` |

#### Environment Variables

| Variable | Required | Description |
| - | - | - |
| `AZURE_OPENAI_API_KEY` | Yes | API key |
| `AZURE_OPENAI_ENDPOINT` | Yes | Endpoint URL |
| `AZURE_OPENAI_API_VERSION` | No | API version |

#### Example

```typescript theme={"theme":"css-variables"}
// Set environment variables first:
// AZURE_OPENAI_API_KEY=...
// AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com

const agent = new HostRunner({
  tools,
  model: "azure/gpt-4o",
  apiKey: process.env.AZURE_OPENAI_API_KEY,
});
```

***

### Amazon Bedrock

| Property | Value |
| - | - |
| Prefix | `bedrock/` |
| Environment Variables | `AWS_BEARER_TOKEN_BEDROCK`, `AWS_REGION` |

Authentication uses a [Bedrock API key](https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html) (bearer token). The AWS region is read from `AWS_REGION` unless a full runtime endpoint is supplied via `baseUrls.bedrock` (e.g. `https://bedrock-runtime.us-east-1.amazonaws.com`).

#### Models

Use any Bedrock model or inference profile ID your AWS account has been granted access to:

| Model ID | Description |
| - | - |
| `us.anthropic.claude-sonnet-4-5-20250929-v1:0` | Claude Sonnet 4.5 (US inference profile) |
| `anthropic.claude-3-5-haiku-20241022-v1:0` | Claude 3.5 Haiku |
| `us.amazon.nova-pro-v1:0` | Amazon Nova Pro |
| `meta.llama3-70b-instruct-v1:0` | Meta Llama 3 70B |

#### Environment Variables

| Variable | Required | Description |
| - | - | - |
| `AWS_BEARER_TOKEN_BEDROCK` | Yes | Bedrock API key (bearer token) |
| `AWS_REGION` | Yes | AWS region (e.g. `us-east-1`), unless `baseUrls.bedrock` is set |

#### Example

```typescript theme={"theme":"css-variables"}
// Set environment variables first:
// AWS_BEARER_TOKEN_BEDROCK=...
// AWS_REGION=us-east-1

const agent = new HostRunner({
  tools,
  model: "bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0",
  apiKey: process.env.AWS_BEARER_TOKEN_BEDROCK,
});
```

***

### Google (Gemini)

| Property | Value |
| - | - |
| Prefix | `google/` |
| Environment Variable | `GOOGLE_GENERATIVE_AI_API_KEY` |

#### Models

| Model ID | Description |
| - | - |
| `gemini-1.5-pro` | Most capable (recommended) |
| `gemini-1.5-flash` | Fast, efficient |
| `gemini-pro` | Original Gemini Pro |

#### Example

```typescript theme={"theme":"css-variables"}
const agent = new HostRunner({
  tools,
  model: "google/gemini-1.5-pro",
  apiKey: process.env.GOOGLE_GENERATIVE_AI_API_KEY,
});
```

***

### Mistral

| Property | Value |
| - | - |
| Prefix | `mistral/` |
| Environment Variable | `MISTRAL_API_KEY` |

#### Models

| Model ID | Description |
| - | - |
| `mistral-large-latest` | Most capable |
| `mistral-medium-latest` | Balanced |
| `mistral-small-latest` | Fast, economical |
| `open-mistral-7b` | Open source |
| `open-mixtral-8x7b` | Open source MoE |

#### Example

```typescript theme={"theme":"css-variables"}
const agent = new HostRunner({
  tools,
  model: "mistral/mistral-large-latest",
  apiKey: process.env.MISTRAL_API_KEY,
});
```

***

### DeepSeek

| Property | Value |
| - | - |
| Prefix | `deepseek/` |
| Environment Variable | `DEEPSEEK_API_KEY` |

#### Models

| Model ID | Description |
| - | - |
| `deepseek-chat` | General chat |
| `deepseek-coder` | Code-focused |

#### Example

```typescript theme={"theme":"css-variables"}
const agent = new HostRunner({
  tools,
  model: "deepseek/deepseek-chat",
  apiKey: process.env.DEEPSEEK_API_KEY,
});
```

***

### Ollama (Local)

| Property | Value |
| - | - |
| Prefix | `ollama/` |
| Default URL | `http://localhost:11434` |

#### Models

Any locally installed model:

* `llama3`
* `codellama`
* `mistral`
* `mixtral`
* etc.

<Note>
  Ensure Ollama is running locally before use. The API key parameter is required but not used, so pass any string.
</Note>

#### Example

```typescript theme={"theme":"css-variables"}
const agent = new HostRunner({
  tools,
  model: "ollama/llama3",
  apiKey: "ollama", // Required but not used
});
```

***

### OpenRouter

| Property | Value |
| - | - |
| Prefix | `openrouter/` |
| Environment Variable | `OPENROUTER_API_KEY` |

Access many models through one API:

| Model ID | Description |
| - | - |
| `anthropic/claude-3-opus` | Claude 3 Opus via OpenRouter |
| `openai/gpt-4-turbo` | GPT-4 Turbo via OpenRouter |
| `google/gemini-pro` | Gemini via OpenRouter |
| `meta-llama/llama-3-70b-instruct` | Llama 3 70B |

#### Example

```typescript theme={"theme":"css-variables"}
const agent = new HostRunner({
  tools,
  model: "openrouter/anthropic/claude-3-opus",
  apiKey: process.env.OPENROUTER_API_KEY,
});
```

***

### xAI (Grok)

| Property | Value |
| - | - |
| Prefix | `xai/` |
| Environment Variable | `XAI_API_KEY` |

#### Models

| Model ID | Description |
| - | - |
| `grok-beta` | Grok beta |

#### Example

```typescript theme={"theme":"css-variables"}
const agent = new HostRunner({
  tools,
  model: "xai/grok-beta",
  apiKey: process.env.XAI_API_KEY,
});
```

***

## Custom Providers

Add your own OpenAI or Anthropic-compatible endpoints.

### CustomProvider Type

```typescript theme={"theme":"css-variables"}
interface CustomProvider {
  name: string;
  protocol: "openai-compatible" | "anthropic-compatible";
  baseUrl: string;
  modelIds: string[];
  useChatCompletions?: boolean;
  apiKeyEnvVar?: string;
}
```

### Properties

| Property | Type | Required | Description |
| - | - | - | - |
| `name` | `string` | Yes | Provider identifier (used in model string) |
| `protocol` | `string` | Yes | `"openai-compatible"` or `"anthropic-compatible"` |
| `baseUrl` | `string` | Yes | API endpoint URL |
| `modelIds` | `string[]` | Yes | Available model IDs |
| `useChatCompletions` | `boolean` | No | Use `/chat/completions` endpoint |
| `apiKeyEnvVar` | `string` | No | Custom env var name for API key |

### OpenAI-Compatible Example

```typescript theme={"theme":"css-variables"}
const agent = new HostRunner({
  tools,
  model: "my-openai/gpt-4",
  apiKey: process.env.MY_API_KEY,
  customProviders: {
    "my-openai": {
      name: "my-openai",
      protocol: "openai-compatible",
      baseUrl: "https://api.my-provider.com/v1",
      modelIds: ["gpt-4", "gpt-3.5-turbo"],
    },
  },
});
```

### Anthropic-Compatible Example

```typescript theme={"theme":"css-variables"}
const agent = new HostRunner({
  tools,
  model: "my-anthropic/claude-3-sonnet",
  apiKey: process.env.MY_API_KEY,
  customProviders: {
    "my-anthropic": {
      name: "my-anthropic",
      protocol: "anthropic-compatible",
      baseUrl: "https://api.my-provider.com",
      modelIds: ["claude-3-sonnet", "claude-3-haiku"],
    },
  },
});
```

### LiteLLM Example

```typescript theme={"theme":"css-variables"}
const agent = new HostRunner({
  tools,
  model: "litellm/gpt-4",
  apiKey: process.env.LITELLM_API_KEY,
  customProviders: {
    litellm: {
      name: "litellm",
      protocol: "openai-compatible",
      baseUrl: "http://localhost:8000",
      modelIds: ["gpt-4", "claude-3-sonnet", "gemini-pro"],
      useChatCompletions: true, // Required for LiteLLM
    },
  },
});
```

***

## Helper Functions

### parseLLMString()

Parse a model string into provider and model ID.

```typescript theme={"theme":"css-variables"}
import { parseLLMString } from "@mcpjam/sdk";

parseLLMString(modelString: string): { provider: string; modelId: string }
```

#### Example

```typescript theme={"theme":"css-variables"}
const { provider, modelId } = parseLLMString("anthropic/claude-sonnet-4-20250514");
// provider: "anthropic"
// modelId: "claude-sonnet-4-20250514"
```

***

### createModelFromString()

Create a provider model instance directly.

```typescript theme={"theme":"css-variables"}
import { createModelFromString } from "@mcpjam/sdk";

createModelFromString(
  llmString: string,
  options: CreateModelOptions
): ProviderLanguageModel
```

#### Example

```typescript theme={"theme":"css-variables"}
const model = createModelFromString(
  "openai/gpt-4o",
  { apiKey: process.env.OPENAI_API_KEY }
);
```

***

### createCustomProvider()

Create a custom provider configuration object.

```typescript theme={"theme":"css-variables"}
import { createCustomProvider } from "@mcpjam/sdk";

createCustomProvider(config: CustomProvider): CustomProvider
```

#### Example

```typescript theme={"theme":"css-variables"}
const provider = createCustomProvider({
  name: "my-litellm",
  protocol: "openai-compatible",
  baseUrl: "http://localhost:8000",
  modelIds: ["gpt-4", "claude-3-sonnet"],
  useChatCompletions: true,
});
```

***

## Environment Variables Summary

| Provider | Variable |
| - | - |
| Anthropic | `ANTHROPIC_API_KEY` |
| OpenAI | `OPENAI_API_KEY` |
| Azure | `AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT` |
| Amazon Bedrock | `AWS_BEARER_TOKEN_BEDROCK`, `AWS_REGION` |
| Google | `GOOGLE_GENERATIVE_AI_API_KEY` |
| Mistral | `MISTRAL_API_KEY` |
| DeepSeek | `DEEPSEEK_API_KEY` |
| OpenRouter | `OPENROUTER_API_KEY` |
| xAI | `XAI_API_KEY` |

***

## Related

* [Testing Across Providers](/sdk/concepts/multi-provider) - Conceptual guide
* [HostRunner Reference](/sdk/reference/host-runner) - Agent configuration


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.