Subset of permissions check on creation

[derek-ho]

Description

This PR adds permission checks around API token creation, and cleans up misc. items from the feature branch, including authorizing API token endpoint properly and stashing the context in order to perform the API token refresh action without explicit permission to do so. This PR is dependent on #4992 to be merged first, and this PR only contains the following changes:

https://github.com/opensearch-project/security/pull/5012/files/aa506e78eb5699d2580e28d4bb0f0060971449e1..e13c055e063ae3e5ee40386a53155e5c65893b58

Issues Resolved

[List any issues this PR will resolve]

Is this a backport? If so, please add backport PR # and/or commits #, and remove backport-failed label from the original PR.

Do these changes introduce new permission(s) to be displayed in the static dropdown on the front-end? If so, please open a draft PR in the security dashboards plugin and link the draft PR here

Testing

[Please provide details of testing done: unit testing, integration testing and manual testing]

Check List

• New functionality includes testing
• New functionality has been documented
• New Roles/Permissions have a corresponding security dashboards plugin PR
• API changes companion pull request created
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Attention: Patch coverage is 76.59574% with 22 lines in your changes missing coverage. Please review.

Project coverage is 71.23%. Comparing base (0edba23) to head (b6aa196).
Report is 33 commits behind head on feature/api-tokens.

Files with missing lines	Patch %	Lines
...arch/security/action/apitokens/ApiTokenAction.java	70.27%	19 Missing and 3 partials ⚠️

Additional details and impacted files

@@                  Coverage Diff                   @@
##           feature/api-tokens    #5012      +/-   ##
======================================================
- Coverage               71.46%   71.23%   -0.24%     
======================================================
  Files                     334      354      +20     
  Lines                   22552    23322     +770     
  Branches                 3590     3692     +102     
======================================================
+ Hits                    16117    16613     +496     
- Misses                   4642     4870     +228     
- Partials                 1793     1839      +46     

Files with missing lines	Coverage Δ
.../opensearch/security/OpenSearchSecurityPlugin.java	83.92% <100.00%> (+0.36%)	⬆️
...ecurity/action/apitokens/ApiTokenIndexHandler.java	84.78% <100.00%> (ø)
.../security/action/apitokens/ApiTokenRepository.java	96.96% <ø> (ø)
...earch/security/auditlog/impl/AbstractAuditLog.java	76.88% <ø> (+0.15%)	⬆️
...rg/opensearch/security/dlic/rest/api/Endpoint.java	100.00% <100.00%> (ø)
...dlic/rest/api/RestApiAdminPrivilegesEvaluator.java	73.07% <100.00%> (+0.52%)	⬆️
...pensearch/security/http/ApiTokenAuthenticator.java	58.97% <ø> (ø)
...arch/security/action/apitokens/ApiTokenAction.java	40.10% <70.27%> (ø)

... and 12 files with indirect coverage changes

[nibix]

So, the token they are requesting is validated to be a subset of the permissions available to them in roles.yml, right?

What is supposed to happen if the permissions in roles.yml are later changed, especially if they are reduced?

[cwperks]

I think the same problem exists with roles as well. If you stored roles information and a user's roles mappings changed then the tokens would need to be invalidated.

[derek-ho]

This is a good point. I added a todo on this issue: #5088 (comment)

[nibix]

Is this assumption safe?

[nibix]

Assume that I have a role with the index_pattern /index_[0-9]+/. How can a requestedPattern look like that replicates this pattern?

At the moment, I have doubts that simple pattern matching can be used to support the full pattern semantics.

[nibix]

This also does not cover alias semantics.

[cwperks]

Assume that I have a role with the index_pattern /index_[0-9]+/. How can a requestedPattern look like that replicates this pattern?

Do role definitions support this? I have only seen this in a context of masked fields

Edit: Found an example: https://github.com/opensearch-project/security/blob/main/src/test/resources/roles.yml#L20-L33

Edit2: Is that the purpose of the renderedMatcher?

Looks like this is handled by WildcardMatcher itself.

[nibix]

What do you think about having a dedicated transport action backing the REST action? Then, one would get access control out of the box without needing explicit checks within the action code.

[cwperks]

Agree with this though it depends on the action.

If the security admin is going to a page in the security dashboards plugin to manage the existing tokens (listing them and performing actions such as revoke) then these should follow the authorization model of other security APIs.

If regular users are allowed to request a token (with a subset of their own permissions) then that could be authorized like other cluster actions in OpenSearch.

[derek-ho]

I think (for now) my operating assumption would be that this is an admin-level action to issue api tokens to connect with other services programmatically. Since API tokens (currently) do not follow the new optimized privileges evaluation, and thus have higher performance penalty than other means of authc/z, and thus we should restrict this as much as possible. However, if there is community feedback/requests for this feature, then we can consider it at a later time. Since we want to authorize these APIs like the others, I added this logic.