Deployment and other DX improvements

[noise64]

Resolves

Resolves #91
Resolves #94
Resolves #163

Manifest:

• HTTP API definitions and HTTP API deployments are now part of the manifest

Deployment:

• HTTP API definitions and HTTP API deployments can be deployed using the app deploy, api deploy, api definition deploy, api deployment deploy commands; the more specific deploy comands also select their dependencies for deployments: e.g. deploying a specific site will also deploy the required components and definitions
• added diffing between server and manifest entites for components, API defs and API deployments
• the diffing creates a unified / normalized representation which is also shown as unified diff when changes are detected
• for file changes (component and IFS) the CLI will calculate hashes, and use diffing on those; the server side files are downloaded during this, but the hashes are cached
• updating non-draft API definitions uses an interactive version editing, which is also saved to the manifest files
• API deployment deletions (removing definitions from it) are handled with interactive confirm question
• all the above is intended to be mostly naïve for now, and expected to be changed as we go forward with atomic deployments in later releases
• redeploy arguments got generalized, using --redeploy-all (-r) or --redeploy-http-api will temporarily undeploy APIs from sites, so version bumps can be avoided during local development

Interactive improvements (non-deployment ones)

• Added command golem component add-dependency for defining component dependencies, when the command is called without arguments it enters an iteractive wizard for creating the dependency
• app new and component new now has an interactive mode, where languages and templates can be selected

Other improvements and fixes

• a new hook is introduced to detect non-running local server, and can auto-start it for the session interactively: we probably want to modify this to only show a warn for how to start the server in general, but for REPL use case i would leave it on, let's clean this up in the next PR
• worker invoke now uses a lenient parse for strings and char WAVE arguments: if a string char does not start with " or a char argument does not start with ' then it is wrapped into those, in other words: no need for double escaping in shells
• decreased the size for sqlite pools, which could resulted in "file open" errors with common setting while using golem server run
• IFS archive handler
  • added HTTP status handling for remote files: non-succesful payloads are not uploaded anymore
  • fixed copying in-memory payloads after download
  • extracted "file processor", so the same traverse can be used for hashing and for zipping
• added a local serve task for app manifest JSON schema in the Makefile.toml that aims to help with testing schema completetions in IDEs locally
• TaskResultMarker rework:
  • identity and inputs are separated now, which fixes some up-to-date check issues in general (e.g. changing dependency types)
  • the result marker now can be used as a "hash cache", which was used for the deployment diffing
• added "app yaml edit" modul for helping with non-destructive app manifest editing
• many error message fixes and improvements, e.g.:
  • support for highlight rib parse errors
  • better format for manifest valudation errors
  • show available options for manifest "foreign key" validation errors (with fuzzy matching)
  • new hint errors that show the clap help after custom messages (e.g. for failed interactive sessions)
• bump required minimum python version to 3.12

Some screenshots

App new

note: component new behaves the same

Add component dependency

Deployment diffs

Error messages

Interactive (re)deployment