UNPKG

@isdk/ai-tool-llm-llamacpp

Version:

llama.cpp LLM Provider

github.com/isdk/ai-tool-llm-llamacpp.js

isdk/ai-tool-llm-llamacpp.js

162 lines (81 loc) • 4.72 kB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
[**@isdk/ai-tool-llm-llamacpp**](../README.md) *** [@isdk/ai-tool-llm-llamacpp](../globals.md) / LLamaCppResult # Interface: LLamaCppResult Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:542](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L542) ## Properties ### chatTemplateId? > `optional` **chatTemplateId**: `object` Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:559](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L559) #### id > **id**: `string` #### version? > `optional` **version**: `string` *** ### content > **content**: `string` Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:543](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L543) *** ### generation\_settings > **generation\_settings**: [`LLamaCppModelGenerationSettings`](LLamaCppModelGenerationSettings.md) Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:544](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L544) *** ### model > **model**: `string` Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:545](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L545) *** ### prompt > **prompt**: `string` Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:546](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L546) *** ### slot\_id > **slot\_id**: `number` Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:547](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L547) *** ### stop > **stop**: `boolean` Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:548](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L548) *** ### stop\_type > **stop\_type**: `"none"` \| `"eos"` \| `"limit"` \| `"word"` Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:558](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L558) *** ### stopped\_eos > **stopped\_eos**: `boolean` Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:549](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L549) *** ### stopped\_limit > **stopped\_limit**: `boolean` Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:550](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L550) *** ### stopped\_word > **stopped\_word**: `boolean` Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:551](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L551) *** ### stopping\_word > **stopping\_word**: `string` Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:552](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L552) *** ### taskId? > `optional` **taskId**: `AsyncTaskId` Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:560](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L560) *** ### timings > **timings**: [`LLamaCppModelTimings`](LLamaCppModelTimings.md) Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:553](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L553) *** ### tokens\_cached > **tokens\_cached**: `number` Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:554](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L554) *** ### tokens\_evaluated > **tokens\_evaluated**: `number` Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:555](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L555) *** ### tokens\_predicted > **tokens\_predicted**: `number` Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:556](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L556) *** ### truncated > **truncated**: `boolean` Defined in: [packages/ai-tool-llm-llamacpp/src/options.ts:557](https://github.com/isdk/ai-tool-llm-llamacpp.js/blob/151b8bdfe7d8b8a8be547948f716da692b2f3c67/src/options.ts#L557)