v0.0.10
Released on August 22, 2022.
Key Features
- Adds support for non-tabular data types; operators can now return any Python-serializable object. Under the hood, Aqueduct has special optimization for JSON blobs, images, and tables, in addition to supporting regular Python objects.
- Enables eager execution when defining workflow artifacts; artifacts are now immediately computed at definition time, before calling the
get
API, which surfaces potential errors earlier during workflow construction.
Enhancements
- Caches previously computed function results to avoid repetitive recomputation.
- Enables using AWS S3 as Aqueduct's metadata store; when connecting an S3 integration, you can now optionally choose to store all Aqueduct metadata in AWS S3.
Bugfixes
- Fixes a bug where the DAG view would ignore the selected version when refreshing the page.
All Changes
All commits
- Eng 1394 allow getting workflow-written/saved objects from the sdk by @eunice-chan in #240
- Remove prints by @eunice-chan in #361
- ENG-1566 Export CSV file name fix by @agiron123 in #355
- Introduce concurrency back into integration tests by @kenxu95 in #349
- Revert integration test concurrency back to 1 by @kenxu95 in #364
- Log server filepaths and line numbers in verbose mode by @kenxu95 in #354
- Remove double workflow deletion from test_delete_workflow_* tests by @eunice-chan in #363
- Implement preview artifact caching by @kenxu95 in #328
- Implement non-tabular type support and eager execution by @cw75 in #344
- Allows config to be modified for running server by @saurav-c in #353
- Adds check for AWS env vars in aqueduct_executor by @hsubbaraj-spiral in #356
- Fix sqlite transactional concurrency issue around SQLITE_BUSY and SQLITE_LOCKED by @kenxu95 in #369
- Add concurrency back to our pre-merge integration tests by @kenxu95 in #370
- Double deletion 2 by @eunice-chan in #372
- Fixes S3 path parsing bug by @saurav-c in #366
- Eng 1522 fix a bug where UI only showing the latest DAG on refresh by @eunice-chan in #375
- Fix data migration script to account for artifact result being missing due to failure by @cw75 in #373
- Fixes noisy bug where database logs err after every txn by @saurav-c in #377
- Remove
show_dag
api from SDK by @cw75 in #378 - Adds backend support for changing storage to S3 integration by @saurav-c in #362
- Adds K8s integration to backend by @hsubbaraj-spiral in #347
- Remove accidentally dumped file by @cw75 in #380
- Fix a minor logging bug during data migration by @cw75 in #381
- Show other artifact types on the UI by @cw75 in #376
- Updates package-lock.json for aqueducthq/common. by @agiron123 in #382
- Make k8s inactive on UI by @hsubbaraj-spiral in #383
- Ready for release 0.0.10 by @cw75 in #384