@isdk/ai-tool-llm-llamacpp-pro/docs/interfaces/LlamaLoadModelOptions.md

llama.cpp LLM Provider

[**@isdk/ai-tool-llm-llamacpp-pro**](../README.md)

***

[@isdk/ai-tool-llm-llamacpp-pro](../globals.md) / LlamaLoadModelOptions

# Interface: LlamaLoadModelOptions

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:230](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L230)

## Extends

- `AIOptions`

## Indexable

\[`name`: `string`\]: `any`

## Properties

### alias?

> `optional` **alias**: `string`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:235](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L235)

***

### batch?

> `optional` **batch**: `number`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:245](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L245)

***

### cache\_type\_k?

> `optional` **cache\_type\_k**: [`LlamaCacheQuantType`](../type-aliases/LlamaCacheQuantType.md)

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:262](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L262)

***

### cache\_type\_v?

> `optional` **cache\_type\_v**: [`LlamaCacheQuantType`](../type-aliases/LlamaCacheQuantType.md)

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:263](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L263)

***

### cont\_batching?

> `optional` **cont\_batching**: `boolean`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:260](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L260)

***

### embedding?

> `optional` **embedding**: `boolean`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:244](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L244)

***

### gpu\_layers?

> `optional` **gpu\_layers**: `number`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:236](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L236)

***

### grp\_attn\_n?

> `optional` **grp\_attn\_n**: `number`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:249](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L249)

***

### grp\_attn\_w?

> `optional` **grp\_attn\_w**: `number`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:250](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L250)

***

### lora?

> `optional` **lora**: [`LoraItem`](../type-aliases/LoraItem.md) \| [`LoraItems`](../type-aliases/LoraItems.md)

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:264](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L264)

***

### lora\_base?

> `optional` **lora\_base**: `string`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:265](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L265)

***

### mmproj?

> `optional` **mmproj**: `string`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:233](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L233)

***

### model

> **model**: `string`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:231](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L231)

***

### n\_ctx?

> `optional` **n\_ctx**: `number`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:243](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L243)

Specify the context window size of the model that you have loaded in your
Llama.cpp server.
// Set the size of the prompt context. The default is 512, but LLaMA models were built with a context of 2048, which will provide better results for longer input/inference.
// The size may differ in other models, for example, baichuan models were build with a context of 4096.

***

### n\_predict?

> `optional` **n\_predict**: `number`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:261](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L261)

***

### n\_threads\_batch?

> `optional` **n\_threads\_batch**: `number`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:248](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L248)

***

### parallel?

> `optional` **parallel**: `number`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:246](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L246)

***

### rope\_freq\_base?

> `optional` **rope\_freq\_base**: `number`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:252](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L252)

***

### rope\_freq\_scale?

> `optional` **rope\_freq\_scale**: `number`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:253](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L253)

***

### rope\_scaling?

> `optional` **rope\_scaling**: [`LlamaRopeScalingType`](../type-aliases/LlamaRopeScalingType.md)

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:251](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L251)

***

### threads?

> `optional` **threads**: `number`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:247](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L247)

***

### use\_mlock?

> `optional` **use\_mlock**: `boolean`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:259](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L259)

***

### use\_mmap?

> `optional` **use\_mmap**: `boolean`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:258](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L258)

***

### yarn\_attn\_factor?

> `optional` **yarn\_attn\_factor**: `number`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:255](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L255)

***

### yarn\_beta\_fast?

> `optional` **yarn\_beta\_fast**: `number`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:256](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L256)

***

### yarn\_beta\_slow?

> `optional` **yarn\_beta\_slow**: `number`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:257](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L257)

***

### yarn\_ext\_factor?

> `optional` **yarn\_ext\_factor**: `number`

Defined in: [packages/ai-tool-llm-llamacpp-pro/src/options.ts:254](https://github.com/isdk/ai-tool-llm-llamacpppro.js/blob/9c0ca0e3e655313e1ab7404ee1b7bd6f7999d23f/src/options.ts#L254)