Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cd/eng 1804/prompt reconciliation for vercel ai sdk #62

Merged
merged 3 commits into from
Aug 28, 2024
Merged

Cd/eng 1804/prompt reconciliation for vercel ai sdk #62

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ea2e5b0
remove unnecessary param
constantinidan Aug 24, 2024
294bc90
add promptID and variables
constantinidan Aug 24, 2024
bd53b2f
remove conversion
constantinidan Aug 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 38 additions & 6 deletions src/instrumentation/vercel-sdk.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ import type {
streamObject,
streamText
} from 'ai';
import { zodToJsonSchema } from 'zod-to-json-schema';

import {
ChatGeneration,
Expand Down Expand Up @@ -61,6 +60,7 @@ const extractMessages = (
const extractSettings = (options: Options<AllVercelFn>): ILLMSettings => {
const settings = { ...options } as any;
delete settings.model;
delete settings.messages;
delete settings.prompt;
delete settings.abortSignal;
if ('tools' in settings) {
Expand All @@ -69,14 +69,12 @@ const extractSettings = (options: Options<AllVercelFn>): ILLMSettings => {
key,
{
description: tool.description,
parameters: zodToJsonSchema(tool.parameters)
parameters: tool.parameters
}
])
);
}
if ('schema' in settings) {
settings.schema = zodToJsonSchema(settings.schema);
}

return settings;
};

Expand All @@ -87,11 +85,27 @@ const extractTools = (options: Options<AllVercelFn>): ITool[] | undefined => {
function: {
name: key,
description: tool.description!,
parameters: zodToJsonSchema(tool.parameters) as any
parameters: tool.parameters
}
}));
};

const extractPrompt = (
options: Options<AllVercelFn>
):
| { uuid: string; promptId: string; variables: Record<string, any> }
| undefined => {
if (!options.messages) return undefined;

for (const message of options.messages) {
if (typeof (message as any).literalMetadata === 'function') {
return (message as any).literalMetadata();
}
Comment on lines +101 to +103
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slightly cleaner style IMO

Suggested change
if (typeof (message as any).literalMetadata === 'function') {
return (message as any).literalMetadata();
}
const metadata = (message as any).literalMetadata();
if (typeof metadata === 'function') {
return metadata();
}

Copy link
Contributor Author

@constantinidan constantinidan Aug 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the suggestion does not work if the messages are not linked to a prompt. .literalMetadata() does not exist.
image

}

return undefined;
};

const computeMetricsSync = (
options: Options<GenerateFn>,
result: Result<GenerateFn>,
Expand Down Expand Up @@ -141,7 +155,16 @@ const computeMetricsSync = (

const messageCompletion = messages.pop();

const promptData = extractPrompt(options);
const promptId = promptData?.promptId;
const variables = promptData?.variables;
if (messages.length > 0) {
messages[0].uuid = promptData?.uuid;
}

return {
promptId,
variables,
duration,
tokenThroughputInSeconds,
outputTokenCount,
Expand Down Expand Up @@ -217,7 +240,16 @@ const computeMetricsStream = async (
if (textMessage.content) messages.push(textMessage);
const messageCompletion = messages.pop();

const promptData = extractPrompt(options);
const promptId = promptData?.promptId;
const variables = promptData?.variables;
if (messages.length > 0) {
messages[0].uuid = promptData?.uuid;
}

return {
promptId,
variables,
duration,
tokenThroughputInSeconds,
outputTokenCount,
Expand Down
Loading