Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report S3 operation name of specific request that failed. #377

Merged
merged 14 commits into from
Nov 22, 2023
Merged

Conversation

graebm
Copy link
Contributor

@graebm graebm commented Nov 22, 2023

Issue:
It was difficult to map errors from aws-c-s3 into errors in other AWS SDKs without knowing exactly which HTTP request failed.

For example, if AWS_S3_META_REQUEST_TYPE_PUT_OBJECT fails, they need to know whether it was a "PutObject", "CreateMultipartUpload", "UploadPart", or "CompleteMultipartUpload" HTTP request that failed.

Description of changes:

  • Report the S3 operation name of the specific HTTP request that failed (e.g. "CreateMultipartUpload").
  • Likewise, report that operation name in metrics.
  • For AWS_S3_META_REQUEST_TYPE_DEFAULT, where aws-c-s3 isn't sure what operation is being performed, allow the user to pass in the operation name.
  • Internally, pass request_type and operation_name around directly. Instead of querying them via vtable gymnastics, which I found confusing and error-prone.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@codecov-commenter
Copy link

codecov-commenter commented Nov 22, 2023

Codecov Report

Merging #377 (77e2a5c) into main (7192ab5) will increase coverage by 0.08%.
The diff coverage is 96.66%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #377      +/-   ##
==========================================
+ Coverage   88.55%   88.64%   +0.08%     
==========================================
  Files          20       20              
  Lines        5348     5380      +32     
==========================================
+ Hits         4736     4769      +33     
+ Misses        612      611       -1     
Files Coverage Δ
source/s3_auto_ranged_get.c 98.26% <100.00%> (+0.64%) ⬆️
source/s3_auto_ranged_put.c 92.67% <ø> (-0.03%) ⬇️
source/s3_copy_object.c 83.05% <ø> (-0.17%) ⬇️
source/s3_default_meta_request.c 94.96% <100.00%> (+0.33%) ⬆️
source/s3_meta_request.c 92.94% <100.00%> (+0.12%) ⬆️
source/s3_util.c 98.79% <100.00%> (+0.11%) ⬆️
source/s3_client.c 88.91% <50.00%> (ø)
source/s3_request.c 95.06% <94.11%> (-0.20%) ⬇️

... and 1 file with indirect coverage changes

@graebm graebm marked this pull request as ready for review November 22, 2023 00:22
source/s3_copy_object.c Outdated Show resolved Hide resolved
int aws_s3_request_metrics_get_operation_name(
const struct aws_s3_request_metrics *metrics,
const struct aws_string **out_operation_name);

/* Get the request type from request metrics. */
AWS_S3_API
void aws_s3_request_metrics_get_request_type(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC we added this function and all the aws_s3_request_type stuff specifically for Mountpoint for use in metrics. If Mountpoint is still the only customer for it and you feel like you want to break it, it would be really easy for us to switch to this new interface instead, since we really only needed it to get operation names anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll keep the enums. I'm finding uses for both enum and string

…them via vtable calls.

In my opinion, the previous approach was complicated, hard to use, and easy to mess up. I didn't realize we weren't handling the case where singlepart operations use DEFAULT request type under the hood. Now our code MUST declare request-type and operation-name up front. It's obvious. It works. Yay.
Instead of weird inconsistency storing NULL for unknown aws_string, but "" for unknown c-string.
source/s3_auto_ranged_put.c Show resolved Hide resolved
source/s3_request.c Outdated Show resolved Hide resolved
source/s3_default_meta_request.c Outdated Show resolved Hide resolved
@@ -510,7 +496,9 @@ static bool s_s3_auto_ranged_put_update(
request = aws_s3_request_new(
meta_request,
AWS_S3_AUTO_RANGED_PUT_REQUEST_TAG_CREATE_MULTIPART_UPLOAD,
0,
AWS_S3_REQUEST_TYPE_CREATE_MULTIPART_UPLOAD,
"CreateMultipartUpload",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

debatable: I would prefer to have a global static string for the operation names we set (I know you are not a fan of those).

  • In case the name changed from the model, it's easier to us to update
  • multiple places sharing the same string.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I eliminated pretty much all the repeatedly typed strings
Still not using global static variables 😛

include/aws/s3/s3_client.h Outdated Show resolved Hide resolved
include/aws/s3/s3_client.h Outdated Show resolved Hide resolved
Co-authored-by: Waqar Ahmed Khan <waahm7@gmail.com>
@graebm graebm merged commit cc6ba34 into main Nov 22, 2023
30 checks passed
@graebm graebm deleted the operation-name branch November 22, 2023 22:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants