update

tamingllms/_config.yml

@@ -3,7 +3,7 @@
 
 title: "Taming Large Language Models: A Practical Guide to LLM Pitfalls with Python Examples"
 author: Tharsis T. P. Souza
-copyright: "2024"  # Copyright year to be placed in the footer
+copyright: "Tharsis T. P. Souza, 2024"  # Copyright year to be placed in the footer
 project: "tamingLLMs" 
 #logo: /home/tobias/src/tamingLLMs/tamingllms/tamingllms/_static/logo.png
 

tamingllms/_toc.yml

@@ -5,9 +5,9 @@ format: jb-book
 root: markdown/toc.md
 chapters:
 - file: markdown/intro.md
-- file: notebooks/nondeterminism.ipynb
 - file: notebooks/output_size_limit.ipynb
 - file: notebooks/structured_output.ipynb
+- file: notebooks/nondeterminism.ipynb
 #- file: nondeterminism-NOTES.md
 #- file: markdown
 #- file: notebooks

tamingllms/notebooks/nondeterminism.ipynb

@@ -5,7 +5,7 @@
    "metadata": {},
    "source": [
     "\n",
-    "# Non-determinism\n",
+    "# Non-determinism & Evals\n",
     "\n",
     "One of the most fundamental challenges when building products with Large Language Models (LLMs) is their non-deterministic nature. Unlike traditional software systems where the same input reliably produces the same output, LLMs can generate different responses each time they're queried - even with identical prompts. This characteristic is both a strength and a significant engineering challenge.\n",
     "\n",
@@ -132,10 +132,17 @@
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Evaluating Non-deterministic \n",
+    "\n",
+    "Evaluating non-deterministic LLM systems can be challenging due to the inherent variability in their responses. Here are some strategies to effectively evaluate these systems.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
    "metadata": {},
-   "outputs": [],
    "source": []
   }
  ],