deprecated unneeded parts of chat iteration code. readme

docs/initial-schema-scope.md → docs/old/initial-schema-scope.md

@@ -1,5 +1,7 @@
 # Initial Schema Scope
 
+This is the initial scope for AgencyAI. This project reuses the codebase for AgencyAI, so this documentation may still be relevant.
+
 This is a SaaS app which is a simple UI for managing and building multiagent AI systems.
 
 Here is a rough outline of the how the app works technically:

docs/message-management-scope.md → docs/old/message-management-scope.md

@@ -1,5 +1,7 @@
 # Message Management
 
+This is the initial scope for AgencyAI. This project reuses the codebase for AgencyAI, so this documentation may still be relevant.
+
 This is a UI for building and managing multiagent AI.
 
 There are multiple agents which will be talking to each other.

docs/setup.md

@@ -0,0 +1,52 @@
+# Setup
+
+0. Open this `readme.md`
+1. `yarn`
+2. `cp src/backend/secrets/.env.defaults src/backend/secrets/.env`
+3. Add environment variables to `.env`
+   1. Use `openssl rand -base64 32` to generate secrets.
+4. `npm run clean-sql`
+5. `npm start`
+6. Mark `./build`, `./src/backend/build`, `./src/common/build`, `./src/web/build` as excluded from IDE analysis.
+7. Mark `./src/build/secrets` as excluded from IDE analysis.
+
+### Deploying
+
+1. Set your custom domain name in `serverless.yml`.
+2. Create a certificate for your domain in [AWS Certificate Manager](https://us-east-1.console.aws.amazon.com/acm/home).
+3. `npm run deploy`
+
+### Installing MySQL (MacOS)
+
+1. `brew install mysql`
+
+### rebuild-aio
+
+`rebuild-aio` is used for several purposes:
+* Building and bundling files into a build folder or artifact which will be zipped and uploaded when deployed.
+* Files can be transpiled during the build process using a custom `scripts/transformer.js` script.
+* During local development, it will monitor file changes, rebuild, and restart the app.
+* It handles graceful restarts and clean shutdowns. See [`rebuild-aio`](https://www.npmjs.com/package/rebuild-aio) documentation for more details.
+* It allows multiple entry points which is useful for starting the main app, but also potentially generating other files.
+
+### Entry Points
+
+There are multiple entry points:
+* serverlessServer.js - The main entry point when the app is deployed. 
+* localServer.js - The main entry point during local development.
+* generateSql.js - Used for generating a `.sql` script for creating tables on fresh startup.
+* generateSchema.js - Generates a `schema.graphql` file in the monorepo root so that IDE tools can be aware of the schema.
+* mysql/localClean.js - This deletes the database so that the next start is fresh.
+
+It is worth noting that the generateSchema.js entry point does not cause the MySQL database to start running.
+In general, it should be possible to import the executable schema from schema.js without causing the database to start.
+The database only starts whenever the first DB query is made.
+When the server.js entry points are used, one of the first things they do is import `migrate.js`, and this is the first DB query.
+
+## Workflows
+
+### Changing the schema
+
+### Creating a resolver
+
+### Creating a MySQL query

readme.md

@@ -1,52 +1,52 @@
-# Setup
+# RAG with Memory
 
-0. Open this `readme.md`
-1. `yarn`
-2. `cp src/backend/secrets/.env.defaults src/backend/secrets/.env`
-3. Add environment variables to `.env`
-   1. Use `openssl rand -base64 32` to generate secrets.
-4. `npm run clean-sql`
-5. `npm start`
-6. Mark `./build`, `./src/backend/build`, `./src/common/build`, `./src/web/build` as excluded from IDE analysis.
-7. Mark `./src/build/secrets` as excluded from IDE analysis.
+This project uses a RAG technique to implement long term memory. There is also an implementation for short term memory. Together, these enable a chatbot which only requires a single chat window for interaction. There are no new chats. Users can simply open the app and begin talking.
 
-### Deploying
+## Use Cases
 
-1. Set your custom domain name in `serverless.yml`.
-2. Create a certificate for your domain in [AWS Certificate Manager](https://us-east-1.console.aws.amazon.com/acm/home).
-3. `npm run deploy`
+This is useful as a convenience in some cases, and in others, it may be crucial.
 
-### Installing MySQL (MacOS)
+### No Chat Management
 
-1. `brew install mysql`
+Some chatbots have the ability to allow users to create new chats. Each chat is independent of the other chats, and this can be useful to prevent the chatbot from getting confused with irrelevant informtion. However, this kind of feature requires the end user to spend time doing chat management. They might have a chat which is dedicated to a certain topic, and they need to dig through past chats to find it. Chat management is a bookkeeping problem.
 
-### rebuild-aio
+As a convenience, a chatbot which only requires a single chat window for interaction is useful. Chat management is no longer a concern for the end user. This can be seen as offloading the bookkeeping work to the AI system.
 
-`rebuild-aio` is used for several purposes:
-* Building and bundling files into a build folder or artifact which will be zipped and uploaded when deployed.
-* Files can be transpiled during the build process using a custom `scripts/transformer.js` script.
-* During local development, it will monitor file changes, rebuild, and restart the app.
-* It handles graceful restarts and clean shutdowns. See [`rebuild-aio`](https://www.npmjs.com/package/rebuild-aio) documentation for more details.
-* It allows multiple entry points which is useful for starting the main app, but also potentially generating other files.
+The benefit new chats provide, that is, starting with a fresh context to avoid confusing the bot, can still be achieved with a single chat window. The user could simply say, "Let's change topics", and the memory system will automatically adjust the context appropriately.
 
-### Entry Points
+### Smart Speakers
 
-There are multiple entry points:
-* serverlessServer.js - The main entry point when the app is deployed. 
-* localServer.js - The main entry point during local development.
-* generateSql.js - Used for generating a `.sql` script for creating tables on fresh startup.
-* generateSchema.js - Generates a `schema.graphql` file in the monorepo root so that IDE tools can be aware of the schema.
-* mysql/localClean.js - This deletes the database so that the next start is fresh.
+For smart speaker devices, there is no UI the user can access to find what they are looking for. Providing a means for interaction in a way that is natural for the user is crucial for enabling the AI with services such as knowledge base management, brainstorming, and providing tailored suggestions.
 
-It is worth noting that the generateSchema.js entry point does not cause the MySQL database to start running.
-In general, it should be possible to import the executable schema from schema.js without causing the database to start.
-The database only starts whenever the first DB query is made.
-When the server.js entry points are used, one of the first things they do is import `migrate.js`, and this is the first DB query.
+## Privacy: Open Source and Edge Computing
 
-## Workflows
+The ability for long term memory systems in AI to collect user data is a privacy concern. A proprietary system which collects mass amounts of user data over many years could be a gold mine for a company but a privacy nightmare for the user, allowing it to do things such as perform sentiment analysis, ad targeting, health predictions, or imitation.
 
-### Changing the schema
+An option for users of AI to be able to use AI without having to worry about their data being collected is to use an open source AI system which runs on edge devices. This is a system which runs on the user's device, and the user has full control over the data. The AI system is privacy aligned and careful to not send personal information to a server, and the user can see the source code and verify that the AI system is not distributing data.
 
-### Creating a resolver
+## Short Term Memory
 
-### Creating a MySQL query
+By giving the chatbot short term, limitations imposed by token limit can be overcome. As a comparison, an implemention of a chatbot that uses a sliding context window where past messages are simply truncated causes the bot to forget the beginning of the conversation, leading to innacurate responses. Important instructions established at the beginning of the conversation may be lost, leading the AI to suddenly behave as if it had never received those instructions. 
+
+Here, a sliding window is still used, but truncated messages are summarized into a section of the context window dedicated to the short term memory summary. If we can fine-tune a model to perform short term memory summarization well, then the AI will be able to remember the important details or instructions of the cnversation, even if the messages where those details were initally introduces have been truncated.
+
+## Long Term Memory
+
+To support features such as knowledge base management, brainstorming, and providing tailored suggestions, a long term memory system is needed. This is a system which can remember information from past conversations and use that information to inform future conversations.
+
+The RAG technique is used to implement long term memory. All user inputs are annotated with vector embeddings, and then when a new user input is received, a section in the context window dedicated to long term memory is updated with relevant information. This section is then used to inform the AI's response.
+
+## Chat Iteration
+
+A single chat iteration is a loop starting with the user inputing a new message, and ending with the AI outputing a response.
+
+When a new message is sent to the chatbot, it will:
+1. Store in chat logs
+2. Prepare the context window
+3. Generate a response
+
+A background process will eventually annotate the newly stored chat message with vector embeddings, so that long term retrieval will have eventual access.
+
+![Context Segmentation](/docs/context-segmentation.png)
+
+![RAG with Memory](/docs/rag-with-memory.png)