Skip to content

Commit

Permalink
Fix splitText (#3653)
Browse files Browse the repository at this point in the history
Co-authored-by: Odilitime <janesmith@airmail.cc>
  • Loading branch information
batudo and odilitime authored Mar 1, 2025
1 parent 05d679e commit da6d60c
Showing 1 changed file with 2 additions and 8 deletions.
10 changes: 2 additions & 8 deletions packages/core/src/generation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1493,19 +1493,13 @@ export function splitText(

while (start < content.length) {
const end = Math.min(start + chunkSize, content.length);

// Ensure we're not creating empty or invalid chunks
if (end > start) {
chunks.push(content.substring(start, end));
}

// Ensure forward progress and prevent infinite loops
const nextStart = end - bleed;
if (nextStart <= start) {
start = end; // If no progress would be made, skip the bleed
} else {
start = nextStart;
}
// Ensure forward progress while preventing infinite loops
start = Math.max(end - bleed, start + 1);
}

return chunks;
Expand Down

0 comments on commit da6d60c

Please sign in to comment.