Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chore/pps-301 #1114

Merged
merged 67 commits into from
Jun 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
8050ea6
initial commit 1
tianj7 Jul 31, 2023
0863ac9
commit 2
tianj7 Aug 3, 2023
e033356
commit 3 with debugging tips
tianj7 Aug 3, 2023
8ca4004
Code rewrites and alembic migration
tianj7 Sep 1, 2023
c9d6b6a
merge master changes
tianj7 Sep 1, 2023
da1599a
add data transformation to migration
tianj7 Sep 6, 2023
3f52313
fix unit tests
tianj7 Sep 6, 2023
690ccef
add unit test and fix migration script
tianj7 Sep 7, 2023
e4a1f43
Add missing logic to data migration, cleanup and comment
tianj7 Sep 7, 2023
37b9a24
add missing old columns
tianj7 Sep 7, 2023
3449f0e
attempt to fix migration error in dev env
tianj7 Sep 8, 2023
c1595c0
address comments and rework migration
tianj7 Sep 19, 2023
c9d8f48
add logging
tianj7 Sep 26, 2023
ea857c0
fix logging
tianj7 Sep 26, 2023
1c4034c
add logging
tianj7 Sep 27, 2023
92c693b
add logging
tianj7 Sep 27, 2023
676ca16
add logging
tianj7 Sep 27, 2023
ad67747
add logging
tianj7 Sep 27, 2023
0ea7093
add logging
tianj7 Sep 28, 2023
94e6320
add logging
tianj7 Sep 28, 2023
8a993b0
add logging
tianj7 Sep 28, 2023
0f67ddd
add logging
tianj7 Sep 29, 2023
c947e61
add logging
tianj7 Sep 29, 2023
ba1cd22
add logging
tianj7 Sep 29, 2023
d6a07d8
fix grant_type accessor
tianj7 Sep 29, 2023
424617b
fix empty array condition
tianj7 Oct 5, 2023
735dbf7
add logging
tianj7 Oct 5, 2023
0b767b4
fix list none check
tianj7 Oct 5, 2023
8cd3110
Merge branch 'master' into chore/PPS-301
tianj7 Oct 5, 2023
f76fe60
remove unnecessary logic and add logging
tianj7 Oct 6, 2023
52169b5
add loggging
tianj7 Oct 6, 2023
c8337ad
add loggging
tianj7 Oct 6, 2023
995e6a7
return form instead of values
tianj7 Oct 6, 2023
2c19235
test out potential fix
tianj7 Oct 6, 2023
f6f59ed
replace authlib valid_token_request function to allow code in url
tianj7 Oct 9, 2023
88d67a7
code clean up and update authutil version
tianj7 Oct 10, 2023
a0a1f4c
Peg flask version to maintain c
tianj7 Oct 10, 2023
7640dc4
peg werkzeug version to work with current tests
tianj7 Oct 10, 2023
394e448
address comments
tianj7 Feb 27, 2024
4eed074
prepare for merging
tianj7 Mar 5, 2024
2b4e714
merge master
tianj7 Mar 5, 2024
20ad2f5
fix unit tests
tianj7 Mar 5, 2024
4837c68
fix unit tests
tianj7 Mar 5, 2024
aff90fb
delete poetry lock before merge
tianj7 Apr 9, 2024
ea11435
merge master
tianj7 Apr 9, 2024
10000fc
add logging
tianj7 Apr 9, 2024
c981a59
add logging
tianj7 Apr 9, 2024
29d715f
test code fix
tianj7 Apr 9, 2024
beaa10a
fix code
tianj7 Apr 9, 2024
1e7daa9
move logic out
tianj7 Apr 9, 2024
2b2d92f
add new logging
tianj7 Apr 9, 2024
05e3846
add logging
tianj7 Apr 10, 2024
98b12a1
add logging
tianj7 Apr 10, 2024
2996c6d
remove excess logging and testing logic
tianj7 Apr 10, 2024
003042f
update implicit grant
tianj7 Apr 18, 2024
157d7bf
add docstring
tianj7 May 28, 2024
a8ae5f1
merge master changes
tianj7 May 28, 2024
da86b3a
delete lock file
tianj7 Jun 11, 2024
7b67bc0
fix bug with joining on None values
tianj7 Jun 17, 2024
4938aef
add poetry.lock
tianj7 Jun 17, 2024
dc3a289
fix bug
tianj7 Jun 17, 2024
c032ddc
add logic to ensure data integrity
tianj7 Jun 17, 2024
a1e6257
add error handling
tianj7 Jun 17, 2024
43083c6
merge master
tianj7 Jun 17, 2024
d4182fd
update lock file
tianj7 Jun 18, 2024
eb62010
testing with a branch of gen3-qa
haraprasadj Jun 27, 2024
10bbac6
Update Jenkinsfile
haraprasadj Jun 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .secrets.baseline
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,7 @@
"filename": "fence/utils.py",
"hashed_secret": "8318df9ecda039deac9868adf1944a29a95c7114",
"is_verified": false,
"line_number": 128
"line_number": 129
}
],
"migrations/versions/a04a70296688_non_unique_client_name.py": [
Expand Down Expand Up @@ -268,7 +268,7 @@
"filename": "tests/conftest.py",
"hashed_secret": "1348b145fa1a555461c1b790a2f66614781091e9",
"is_verified": false,
"line_number": 1559
"line_number": 1561
},
{
"type": "Base64 High Entropy String",
Expand Down
4 changes: 4 additions & 0 deletions bin/fence_create.py
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,9 @@ def parse_arguments():
help='scopes to include in the token (e.g. "user" or "data")',
)
token_create.add_argument("--exp", help="time in seconds until token expiration")
token_create.add_argument(
tianj7 marked this conversation as resolved.
Show resolved Hide resolved
"--client_id", help="Client Id, required to generate refresh token"
)

force_link_google = subparsers.add_parser("force-link-google")
force_link_google.add_argument(
Expand Down Expand Up @@ -581,6 +584,7 @@ def main():
username=args.username,
scopes=args.scopes,
expires_in=args.exp,
client_id=args.client_id,
)
token_type = str(args.type).strip().lower()
if token_type == "access_token" or token_type == "access":
Expand Down
26 changes: 5 additions & 21 deletions fence/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
import flask
from flask_cors import CORS
from sqlalchemy.orm import scoped_session
from flask import _app_ctx_stack, current_app
from flask import current_app
from werkzeug.local import LocalProxy

from authutils.oauth2.client import OAuthClient
Expand Down Expand Up @@ -364,7 +364,6 @@ def app_config(
_setup_audit_service_client(app)
_setup_data_endpoint_and_boto(app)
_load_keys(app, root_dir)
_set_authlib_cfgs(app)

app.prometheus_counters = {}
if config["ENABLE_PROMETHEUS_METRICS"]:
Expand Down Expand Up @@ -406,24 +405,6 @@ def _load_keys(app, root_dir):
}


def _set_authlib_cfgs(app):
# authlib OIDC settings
# key will need to be added
settings = {"OAUTH2_JWT_KEY": keys.default_private_key(app)}
app.config.update(settings)
config.update(settings)

# only add the following if not already provided
config.setdefault("OAUTH2_JWT_ENABLED", True)
config.setdefault("OAUTH2_JWT_ALG", "RS256")
config.setdefault("OAUTH2_JWT_ISS", app.config["BASE_URL"])
config.setdefault("OAUTH2_PROVIDER_ERROR_URI", "/api/oauth2/errors")
app.config.setdefault("OAUTH2_JWT_ENABLED", True)
app.config.setdefault("OAUTH2_JWT_ALG", "RS256")
app.config.setdefault("OAUTH2_JWT_ISS", app.config["BASE_URL"])
app.config.setdefault("OAUTH2_PROVIDER_ERROR_URI", "/api/oauth2/errors")


def _setup_oidc_clients(app):
configured_idps = config.get("OPENID_CONNECT", {})

Expand Down Expand Up @@ -481,7 +462,10 @@ def _setup_oidc_clients(app):
logger=logger,
)
elif idp == "fence":
app.fence_client = OAuthClient(**settings)
# https://docs.authlib.org/en/latest/client/flask.html#flask-client
app.fence_client = OAuthClient(app)
# https://docs.authlib.org/en/latest/client/frameworks.html
app.fence_client.register(**settings)
else: # generic OIDC implementation
client = Oauth2ClientBase(
settings=settings,
Expand Down
34 changes: 20 additions & 14 deletions fence/blueprints/login/fence_login.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,19 +30,22 @@ def __init__(self):

def get(self):
"""Handle ``GET /login/fence``."""
oauth2_redirect_uri = flask.current_app.fence_client.client_kwargs.get(
"redirect_uri"
)

# OAuth class can have mutliple clients
client = flask.current_app.fence_client._clients[
flask.current_app.config["OPENID_CONNECT"]["fence"]["name"]
]

oauth2_redirect_uri = client.client_kwargs.get("redirect_uri")

redirect_url = flask.request.args.get("redirect")
if redirect_url:
validate_redirect(redirect_url)
flask.session["redirect"] = redirect_url
(
authorization_url,
state,
) = flask.current_app.fence_client.generate_authorize_redirect(
oauth2_redirect_uri, prompt="login"
)

rv = client.create_authorization_url(oauth2_redirect_uri, prompt="login")

authorization_url = rv["url"]

# add idp parameter to the authorization URL
if "idp" in flask.request.args:
Expand All @@ -57,7 +60,7 @@ def get(self):
flask.session["shib_idp"] = shib_idp
authorization_url = add_params_to_uri(authorization_url, params)

flask.session["state"] = state
flask.session["state"] = rv["state"]
return flask.redirect(authorization_url)


Expand Down Expand Up @@ -88,16 +91,19 @@ def get(self):
" login page for the original application to continue."
)
# Get the token response and log in the user.
redirect_uri = flask.current_app.fence_client._get_session().redirect_uri
tokens = flask.current_app.fence_client.fetch_access_token(
redirect_uri, **flask.request.args.to_dict()
client_name = config["OPENID_CONNECT"]["fence"].get("name", "fence")
client = flask.current_app.fence_client._clients[client_name]
oauth2_redirect_uri = client.client_kwargs.get("redirect_uri")

tokens = client.fetch_access_token(
oauth2_redirect_uri, **flask.request.args.to_dict()
)

try:
# For multi-Fence setup with two Fences >=5.0.0
id_token_claims = validate_jwt(
tokens["id_token"],
aud=self.client.client_id,
aud=client.client_id,
scope={"openid"},
purpose="id",
attempt_refresh=True,
Expand Down
5 changes: 4 additions & 1 deletion fence/blueprints/login/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,10 @@ def allowed_login_redirects():
with flask.current_app.db.session as session:
clients = session.query(Client).all()
for client in clients:
allowed.extend(client.redirect_uris)
if isinstance(client.redirect_uris, list):
allowed.extend(client.redirect_uris)
elif isinstance(client.redirect_uris, str):
allowed.append(client.redirect_uris)
return {domain(url) for url in allowed}


Expand Down
16 changes: 14 additions & 2 deletions fence/blueprints/oauth2.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,14 @@
from fence.utils import clear_cookies
from fence.user import get_current_user
from fence.config import config

from authlib.oauth2.rfc6749.errors import (
InvalidScopeError,
)
from fence.utils import validate_scopes
from cdislogging import get_logger

blueprint = flask.Blueprint("oauth2", __name__)
logger = get_logger(__name__)


@blueprint.route("/authorize", methods=["GET", "POST"])
Expand Down Expand Up @@ -114,14 +119,21 @@ def authorize(*args, **kwargs):
return flask.redirect(login_url)

try:
grant = server.validate_consent_request(end_user=user)
grant = server.get_consent_grant(end_user=user)
except OAuth2Error as e:
raise Unauthorized("Failed to authorize: {}".format(str(e)))

client_id = grant.client.client_id
with flask.current_app.db.session as session:
client = session.query(Client).filter_by(client_id=client_id).first()

# Need to do scope check here now due to our design of putting allowed_scope on client
# Authlib now put allowed scope on OIDC server side which doesn't work with our design without modification to the lib
# Doing the scope check here because both client and grant is available here
# Either Get or Post request
request_scopes = flask.request.args.get("scope") or flask.request.form.get("scope")
validate_scopes(request_scopes, client)

# TODO: any way to get from grant?
confirm = flask.request.form.get("confirm") or flask.request.args.get("confirm")
if client.auto_approve:
Expand Down
17 changes: 5 additions & 12 deletions fence/config-default.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,9 @@ OPENID_CONNECT:
# If this fence instance is a client of another fence, fill this cfg out.
# REMOVE if not needed
fence:
# Custom name to display for consent screens. If not provided, will use `fence`.
# If the other fence is using NIH Login, you should make name: `NIH Login`
name: ''
tianj7 marked this conversation as resolved.
Show resolved Hide resolved
# this api_base_url should be the root url for the OTHER fence
# something like: https://example.com
api_base_url: ''
Expand All @@ -155,9 +158,6 @@ OPENID_CONNECT:
authorize_url: '{{api_base_url}}/oauth2/authorize'
access_token_url: '{{api_base_url}}/oauth2/token'
refresh_token_url: '{{api_base_url}}/oauth2/token'
# Custom name to display for consent screens. If not provided, will use `fence`.
# If the other fence is using NIH Login, you should make name: `NIH Login`
name: ''
# if mock is true, will fake a successful login response for login
# WARNING: DO NOT ENABLE IN PRODUCTION (for testing purposes only)
mock: false
Expand Down Expand Up @@ -386,16 +386,9 @@ ENABLED_IDENTITY_PROVIDERS: {}


# //////////////////////////////////////////////////////////////////////////////////////
# LIBRARY CONFIGURATION (authlib & flask)
# LIBRARY CONFIGURATION (flask)
# - Already contains reasonable defaults
# //////////////////////////////////////////////////////////////////////////////////////
# authlib-specific configs for OIDC flow and JWTs
# NOTE: the OAUTH2_JWT_KEY cfg gets set automatically by fence if keys are setup
# correctly
OAUTH2_JWT_ALG: 'RS256'
OAUTH2_JWT_ENABLED: true
OAUTH2_JWT_ISS: '{{BASE_URL}}'
OAUTH2_PROVIDER_ERROR_URI: '/api/oauth2/errors'

# used for flask, "path mounted under by the application / web server"
# since we deploy as microservices, fence is typically under {{base}}/user
Expand Down Expand Up @@ -691,7 +684,7 @@ GS_BUCKETS: {}
# bucket3:
# region: 'us-east-1'

# When using the Cleversafe storageclient, whether or not to send verify=true
# When using the Cleversafe storageclient, whether or not to send verify=true
# for requests
VERIFY_CLEVERSAFE_CERT: true

Expand Down
Loading
Loading