-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Use GToolkit-specific package for LLM core [feenkcom/gtoolkit#4322]
- Loading branch information
Showing
107 changed files
with
849 additions
and
762 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
...Llm/GtBPEDemoEncodingResultProxy.class.st → ...kit/GtBPEDemoEncodingResultProxy.class.st
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
src/Gt4Llm/GtBPEDemoMergeProxy.class.st → ...Llm-GToolkit/GtBPEDemoMergeProxy.class.st
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
Extension { #name : #GtBPEEncoder } | ||
|
||
{ #category : #'*Gt4Llm-GToolkit' } | ||
GtBPEEncoder >> getStatsFor: aListOfTokens [ | ||
| stats | | ||
stats := GtCounter new. | ||
1 | ||
to: aListOfTokens size - 1 | ||
do: [ :anIndex | | ||
stats | ||
increment: {aListOfTokens at: anIndex. | ||
aListOfTokens at: anIndex + 1} ]. | ||
^ stats | ||
] |
2 changes: 1 addition & 1 deletion
2
src/Gt4Llm/GtHome.extension.st → src/Gt4Llm-GToolkit/GtHome.extension.st
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
Extension { #name : #GtLlmActionChat } | ||
|
||
{ #category : #'*Gt4Llm-GToolkit' } | ||
GtLlmActionChat >> asViewModel [ | ||
^ GtLlmActionChatViewModel new assistantChat: self | ||
] | ||
|
||
{ #category : #'*Gt4Llm-GToolkit' } | ||
GtLlmActionChat >> executeWait: aBlock [ | ||
(self whenReadyFuture then: [ :_ | aBlock value: self messages last ]) await | ||
] |
2 changes: 1 addition & 1 deletion
2
src/Gt4Llm/GtLlmActionChatViewModel.class.st → ...Toolkit/GtLlmActionChatViewModel.class.st
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
Extension { #name : #GtLlmActionsGroup } | ||
|
||
{ #category : #'*Gt4Llm-GToolkit' } | ||
GtLlmActionsGroup >> gtActionsFor: aView [ | ||
<gtView> | ||
^ aView columnedList | ||
title: 'Actions'; | ||
priority: 1; | ||
items: [ self items ]; | ||
column: 'Name' | ||
text: #name | ||
weight: 0.5; | ||
column: 'Description' text: #description; | ||
column: 'Actions' | ||
stencil: [ :anItem | | ||
BrButton new | ||
aptitude: BrGlamorousButtonWithIconAptitude; | ||
icon: BrGlamorousVectorIcons cancel; | ||
label: 'Remove'; | ||
action: [ :aButton | | ||
self remove: anItem. | ||
anItem removeFromSystem. | ||
aButton phlow fireToolUpdateWish ] ] | ||
width: 70 | ||
] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
Extension { #name : #GtLlmChat } | ||
|
||
{ #category : #'*Gt4Llm-GToolkit' } | ||
GtLlmChat >> asViewModel [ | ||
^ GtLlmChatViewModel new assistantChat: self | ||
] | ||
|
||
{ #category : #'*Gt4Llm-GToolkit' } | ||
GtLlmChat >> gtMessagesWidgetFor: aView [ | ||
"cannot be a forward because the update wouldn’t work" | ||
|
||
<gtView> | ||
| aViewModel | | ||
self provider ifNil: [ ^ aView empty ]. | ||
|
||
^ aView explicit | ||
title: 'Chat'; | ||
priority: 0; | ||
stencil: [ aViewModel ifNil: [ aViewModel := self asViewModel ]. | ||
GtLlmChatElement new chatViewModel: aViewModel ]; | ||
actionUpdateButton | ||
] | ||
|
||
{ #category : #'*Gt4Llm-GToolkit' } | ||
GtLlmChat >> gtWriteFor: aView [ | ||
<gtView> | ||
^ aView textEditor | ||
title: 'Write'; | ||
priority: 4; | ||
text: [ 'Write a message here' ]; | ||
actionButtonIcon: BrGlamorousVectorIcons play | ||
tooltip: 'Send' | ||
action: [ :aButton | | ||
aButton parent parent parent parent | ||
allChildrenBreadthFirstDetect: [ :anElement | anElement isKindOf: BrEditor ] | ||
ifFound: [ :anEditor | | ||
self sendMessage: anEditor text asString. | ||
anEditor text: '' ] ] | ||
] | ||
|
||
{ #category : #'*Gt4Llm-GToolkit' } | ||
GtLlmChat >> whenReadyFuture [ | ||
^ GtLlmChatReadyFuture new chat: self | ||
] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
src/Gt4Llm/GtLlmChatExamples.class.st → ...t4Llm-GToolkit/GtLlmChatExamples.class.st
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
Extension { #name : #GtLlmChatMessage } | ||
|
||
{ #category : #'*Gt4Llm-GToolkit' } | ||
GtLlmChatMessage >> asViewModel [ | ||
^ GtLlmThreadMessageViewModel new threadMessage: self | ||
] | ||
|
||
{ #category : #'*Gt4Llm-GToolkit' } | ||
GtLlmChatMessage >> gtContentFor: aView [ | ||
<gtView> | ||
<gtLlmMessageView> | ||
^ aView explicit | ||
title: 'Content'; | ||
priority: 5; | ||
stencil: [ GtLlmThreadMessageElement new | ||
threadMessageViewModel: self asViewModel ] | ||
] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
Extension { #name : #GtLlmEmbeddingRegistry } | ||
|
||
{ #category : #'*Gt4Llm-GToolkit' } | ||
GtLlmEmbeddingRegistry >> gtEmbeddingsFor: aView [ | ||
<gtView> | ||
^ aView columnedList | ||
title: 'Embeddings'; | ||
priority: 1; | ||
items: [ embeddings associations ]; | ||
column: 'Input' text: [ :aPair | aPair value input ]; | ||
column: 'Actions' | ||
stencil: [ :aPair | | ||
BrButton new | ||
aptitude: BrGlamorousButtonWithIconAptitude; | ||
label: 'Remove'; | ||
icon: BrGlamorousVectorIcons remove; | ||
action: [ :aButton | | ||
embeddings removeKey: aPair key. | ||
aButton phlow fireUpdateWish ] ] | ||
width: 80; | ||
send: #value | ||
] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
17 changes: 17 additions & 0 deletions
17
src/Gt4Llm-GToolkit/GtLlmFailureThreadMessage.extension.st
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
Extension { #name : #GtLlmFailureThreadMessage } | ||
|
||
{ #category : #'*Gt4Llm-GToolkit' } | ||
GtLlmFailureThreadMessage >> elementClass [ | ||
^ GtLlmThreadMessageElement | ||
] | ||
|
||
{ #category : #'*Gt4Llm-GToolkit' } | ||
GtLlmFailureThreadMessage >> gtContentFor: aView [ | ||
<gtView> | ||
<gtLlmMessageView> | ||
^ aView explicit | ||
title: 'Content'; | ||
priority: 1; | ||
stencil: [ GtLlmThreadMessageElement new | ||
threadMessageViewModel: (GtLlmThreadMessageViewModel new threadMessage: self) ] | ||
] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
Extension { #name : #GtLlmFineTuningFile } | ||
|
||
{ #category : #'*Gt4Llm-GToolkit' } | ||
GtLlmFineTuningFile >> costsForEpochs: anInteger [ | ||
| numberOfTokens | | ||
numberOfTokens := self numberOfTokens * anInteger. | ||
|
||
"this model is free until september 23rd 2024 for under 2M tokens" | ||
((self model beginsWith: 'gpt-4o-mini') | ||
and: [ Date today | ||
< (Date | ||
year: 2024 | ||
month: 9 | ||
day: 23) and: [ numberOfTokens < 2000000 ] ]) | ||
ifTrue: [ ^ GtTCurrencyMoney new | ||
amount: 0; | ||
currency: GtTCurrency usd ]. | ||
|
||
^ GtTCurrencyMoney new | ||
amount: numberOfTokens * self pricePerToken; | ||
currency: GtTCurrency usd | ||
] | ||
|
||
{ #category : #'*Gt4Llm-GToolkit' } | ||
GtLlmFineTuningFile >> costsPerEpoch [ | ||
| numberOfTokens | | ||
numberOfTokens := self numberOfTokens. | ||
|
||
"this model is free until september 23rd 2024 for under 2M tokens" | ||
((self model beginsWith: 'gpt-4o-mini') | ||
and: [ Date today | ||
< (Date | ||
year: 2024 | ||
month: 9 | ||
day: 23) and: [ numberOfTokens < 2000000 ] ]) | ||
ifTrue: [ ^ GtTCurrencyMoney new | ||
amount: 0; | ||
currency: GtTCurrency usd ]. | ||
|
||
^ GtTCurrencyMoney new | ||
amount: numberOfTokens * self pricePerToken; | ||
currency: GtTCurrency usd | ||
] | ||
|
||
{ #category : #'*Gt4Llm-GToolkit' } | ||
GtLlmFineTuningFile >> gtCostFor: aView [ | ||
<gtView> | ||
^ aView forward | ||
title: 'Cost'; | ||
priority: 3; | ||
object: [ self costsPerEpoch ]; | ||
view: #gtDetailsFor: | ||
] | ||
|
||
{ #category : #'*Gt4Llm-GToolkit' } | ||
GtLlmFineTuningFile >> numberOfTokens [ | ||
^ (GtLlmTokenizer new | ||
tokenizeMessages: (conversations flatCollect: #items) | ||
usingModel: self model) tokenCount | ||
] |
2 changes: 1 addition & 1 deletion
2
src/Gt4Llm/GtLlmInspectorSummarizer.class.st → ...Toolkit/GtLlmInspectorSummarizer.class.st
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.