All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- Simplify exception capture inside return_handler to avoid unique, unexpected behavior. Remove debug=True cast of unknown exceptions to FailCatastrophically
- Prevent structlogger from prepending timestamp
- Log FailCatastrophically exceptions as they happen instead of all at once
- Change default parameter passed to functions from
event
tostate
- Call exception_handler on FailCatastrophically
- Fix bug in tests caused by change of default_kwargs["event"] to namedtuple
- format
- Allow user of lpipe to process_event with a callable/function and avoid all directed-graph workflow setup
- Change Payload to split paths into paths and queues. Update all logic to assume only path strings / enums are in payload.paths.
- Fix bug in get_nested where head could result in incorrectly returne default value
- Add more documentation
- Make 'path' argument optional on the
Queue
class, and updatepipeline.put_record
to send kwargs at the root of the message if no path was set.
- Added contrib imports to
contrib/__init__.py
- Reduced code complexity
- Add type hints
- Increase test coverage
- Bugfix: lpipe.sqs.delete_message_batch called boto3 incorrectly
- Breaking: sentry-sdk no longer a required dependency, install with
lpipe = {extras: ["sentry"], version: "*"}
, and enable it by settingprocess_event(exception_handler=lpipe.contrib.sentry.capture)
- Breaking: moved
lpipe.sentry
tolpipe.contrib.sentry
- Breaking: moved
lpipe._boto3
tolpipe.contrib.boto3
- Breaking: removed
lpipe.testing
, use the boto3-fixtures library instead - Breaking: removed
lpipe.taxonomy
- Breaking: renamed LpipeBaseException to LPBaseException (although, you shouldn't use the base class anyways)
- Breaking: renamed ServerlessLogger to LPLogger
- Breaking: renamed
lpipe.utils._repr
torepr
- Semi-Breaking: moved Action, Payload, and Queue classes into their own modules, but they're still imported into the top level module (i.e.
lpipe.Action
still works) - Deprecated: moved
lpipe.utils
get_nested, set_nested tolpipe.contrib.mindictive
, stubs still in place
- To prevent log explosion, stop adding message kwargs by default to logger passed into user's functions.
- Set lpipe.testing.awslambda default lambda name to "my_lambda"
- Fix formatting and linting errors
- SQS: If any records fail, delete successful records from the queue before raising an exception.
- Fix bug in
lpipe._boto.proxy
where logger was undefined.
- If
AWS_ENDPOINTS
is set with a string as dict ("key=value,key=value"
) of service names and endpoints, create boto3 clients and resources with the appropriateendpoint_url
for a service, if provided.
- Add setup/teardown context managers for all AWS testing utils.
- Update get_queue_arn() to request QueueArn specifically.
- Update SQS testing create utils to create DLQs and redrive policies automatically.
- Fix reference to event_source_arn in get_event_source()
- Fix bug in
utils.get_enum_path()
when re-raising an error after catching a KeyError.
- Print exception class dynamically when catching TypeError.
- Pass entire payload via event rather than just the event_source_arn.
utils.get_nested()
now has the option of setting a default value, and it will raise a KeyError if the requested keys don't exist.- Unhandled exceptions will now be raised if debug is true.
- Move event and context into a single dict pased to functions as kwarg
event
. - Pass event source ARN to Payload and, subsequently, to the event dict passed into functions.
- Add ability to pass kwargs even if they aren't in the signature
- Refactor most tests to just use moto.
- Add s3 testing helpers
- Backfill CHANGELOG
- Pass event and context to every function
context
is now a required argument ofprocess_event()
- Allow
process_event()
without apath_enum
/Enum
. One will be automatically generated based on thepaths
provided. - Refactor and document exception flow control. All lpipe errors will now inherit from
LpipeBaseException
. - Move setup and tear-down testing utilities out of
conftest.py
and intolpipe.testing
so they can be used consistently by all projects implementing lpipe. - Add example of using moto to mock boto3.
- Allow user to set
default_path
. Doing so will ignore all message structure and pass the entire message to the called function.
- Add check_status to utils
utils.get_nested
will attempt a getattr() call if HEAD isn't a dict.
- Add
utils.set_nested
dict helper.
- Add JSON encoder
utils.AutoEncoder
which handles encoding Enums.
- Fix bug in ServerlessLogger where we were modifying the base class and emit test logs with context
- Add ability to trigger pipeline directly with a QueueType.RAW which may be a dict or json string
- Add ability to manually return a value for each action triggered by a message.
- Add optional context to sentry.init
- Better dummy lambda venv install command pulled from the build harness
- Capture unhandled errors with Sentry, if it is initialized.
- Enable validation of function signatures where the default is None.
- Pop logger:None off of action_kwargs before continuing so we don't get crazy kwargs during debug/testing.
- Increase verbosity of InvalidInputError messages.
- Remove black from the env; it should be installed locally atm.
- Fix date in CHANGELOG
- Switched function call back to nested dictionary expansion to fix "f() got multiple values for keyword argument 'logger'"
- Add kinesis build() tests matching those added for sqs in 1.0.1
- Updated lpipe.sqs.build to accept message_group_id
- Updated the function signatures of kinesis and sqs helpers for single and batch messages/records.
- Assert Action() functions are instance of types.FunctionType
- Validate event kwargs against function signatures, making required_params optional if a function is provided.
- Validate function signature type hints and defaults.
- required_params only required if no functions are provided
- Only functions or paths are required by Action() constructor (no more blank constructor arguments)