Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backend version updates #605

Merged
merged 22 commits into from
Feb 20, 2024
Merged

Backend version updates #605

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
b5a171b
first pass
rm03 Jan 6, 2024
f2e5528
configure ruff
rm03 Jan 16, 2024
8f3dd99
pin black version
rm03 Jan 16, 2024
86fd1ed
update Python version in backend check, revert pin black version
rm03 Jan 18, 2024
363860d
fix..?
rm03 Jan 21, 2024
97bc979
switch to Python 3.11, fix tests
rm03 Jan 22, 2024
3e00d64
fix dependency
rm03 Jan 22, 2024
f23ff2d
Merge branch 'master' into version-upgrades
rm03 Jan 22, 2024
eec2172
lint
rm03 Jan 22, 2024
74d67e1
Merge branch 'master' into version-upgrades
rm03 Feb 12, 2024
e458eaf
fix channels
rm03 Feb 17, 2024
ae2dbd4
use ruff w/shared actions, remove black, flake8, isort
rm03 Feb 18, 2024
ba8c264
oops lol
rm03 Feb 18, 2024
b712858
this is embarrassing
rm03 Feb 18, 2024
682b1d6
use 3.11 base image
rm03 Feb 18, 2024
d8fe2fd
temporarily add integration tests
rm03 Feb 18, 2024
0f8b271
attempt to fix
rm03 Feb 19, 2024
c75d077
fixed django base
rm03 Feb 19, 2024
f2df40b
fix integration tests
rm03 Feb 20, 2024
b9fde49
fix csrf?
rm03 Feb 20, 2024
58349c1
plz
rm03 Feb 20, 2024
659d62f
remove integration tests
rm03 Feb 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/build-and-deploy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ jobs:
with:
projectName: pennclubs
path: backend
pythonVersion: 3.11-bookworm
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this bookworm stuff?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bookworm is the latest stable Debian release and it seems that Docker has removed support for buster (which is two versions behind bookworm) with later versions of Python. Also if we don't specify the image it defaults to python3.8-buster which doesn't make sense now that we're using 3.11

flake: true
black: true

Expand Down
20 changes: 6 additions & 14 deletions .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,29 +2,21 @@
repos:
- repo: local
hooks:
- id: black
name: black
entry: bash -c "export PIPENV_IGNORE_VIRTUALENVS=1 && cd backend && pipenv run black ."
- id: ruff-format
name: Ruff Formatter
entry: bash -c "export PIPENV_IGNORE_VIRTUALENVS=1 && cd backend && pipenv run ruff format ."
language: python
types: [python]
require_serial: true
files: ^backend/
pass_filenames: false
- id: isort
name: isort
entry: isort
- id: ruff-lint-fix
name: Ruff Lint (Apply Fixes)
entry: ruff check . --fix
language: python
types: [python]
require_serial: true
files: ^backend/
- id: flake8
name: flake8
entry: flake8
language: python
types: [python]
require_serial: true
files: ^backend/
args: [--config, backend/setup.cfg]
- id: frontend
name: Yarn Linter
entry: bash -c "cd frontend && yarn lint"
Expand Down
2 changes: 1 addition & 1 deletion backend/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM pennlabs/django-base:a142aa6975ee293bbc8a09ef0b81998ce7063dd3
FROM pennlabs/django-base:latest

LABEL maintainer="Penn Labs"

Expand Down
15 changes: 9 additions & 6 deletions backend/Pipfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,21 @@ url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]
black = "==19.10b0"
black = "*"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this still here? And why is it = "*"? (Same for flake8)

Copy link
Member Author

@rm03 rm03 Feb 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be mistaken, but I think the CI expects black and flake8 to be installed as part of the Django check, so until we migrate shared-actions to use ruff we should keep these here? I removed the version pins as I wasn't sure why they were necessary

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just updated shared-actions to use ruff, and got rid of black, flake8, and isort

unittest-xml-reporting = ">=3.0.2"
flake8 = "==5.0.3"
flake8 = "*"
flake8-isort = "*"
isort = "*"
flake8-quotes = "*"
django-debug-toolbar = "*"
django-extensions = "*"
ruff = "*"

[packages]
dj-database-url = "*"
djangorestframework = "*"
sentry-sdk = "*"
django = "<4"
django = ">=5"
django-labs-accounts = "*"
drf-nested-routers = "*"
"bs4" = "*"
Expand All @@ -34,7 +35,7 @@ qrcode = "*"
python-dateutil = "*"
psycopg2 = "*"
django-simple-history = "*"
channels = "<3"
channels = "*"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you just confirm we're not depending on a version pin here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this was the only part of our code that needed to be updated, but I'll do further checks/tests locally to make sure unpinning the version doesnt cause any issues

django-runtime-options = "*"
social-auth-app-django = "*"
django-redis = "*"
Expand All @@ -44,7 +45,7 @@ uvloop = {version = "*", sys_platform = "== 'linux'"}
uvicorn = {extras = ["standard"], version = "*"}
gunicorn = "*"
httptools = "*"
ics = "==0.7"
ics = "*"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you just confirm we're not depending on a version pin here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was only one apparent issue with imports relating to the version pin which I fixed here

urlextract = "*"
jsonref = "*"
unittest-xml-reporting = "*"
Expand All @@ -57,6 +58,8 @@ pandas = "*"
drf-excel = "*"
numpy = "*"
coverage = "*"
daphne = "*"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain why we need daphne and tatsu, and why tatsu needs a pin?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

daphne is needed because channels.testing now requires it but doesn't install it as a subdependency (see this)

I think we should be able to get rid of the tatsu pin. I don't remember why I added it (could've been because they dropped support for Python 3.10, but this isn't relevant as we're using 3.11 anyway).

Copy link
Member Author

@rm03 rm03 Feb 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember why I added the tatsu pin now, it's because the latest Django base image uses Python 3.10. Will try and use an image with Python 3.11

tatsu = "==5.8.3"

[requires]
python_version = "3"
python_version = "3.11"
2,697 changes: 1,337 additions & 1,360 deletions backend/Pipfile.lock
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion backend/clubs/management/commands/import_paideia_events.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ def handle(self, *args, **kwargs):
"https://snfpaideia.upenn.edu/wp-json/wp/v2/event?"
"order=asc&per_page=100&page=1&onlyInclude=upcoming_events"
)
with (urllib.request.urlopen(url)) as data:
with urllib.request.urlopen(url) as data:
parsed_json = json.loads(data.read().decode())
for event in parsed_json:
try:
Expand Down
3 changes: 2 additions & 1 deletion backend/clubs/management/commands/merge_duplicates.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,8 @@ def merge_clubs(one, two):
Membership.objects.filter(club=primary).values_list("person__id", flat=True)
)
Membership.objects.filter(
club=secondary, person__in=duplicate_memberships,
club=secondary,
person__in=duplicate_memberships,
).delete()
Membership.objects.filter(club=secondary).update(club=primary)

Expand Down
5 changes: 4 additions & 1 deletion backend/clubs/management/commands/populate.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -672,7 +672,10 @@ def get_image(url):
status_counter % len(ApplicationSubmission.STATUS_TYPES)
][0]
ApplicationSubmission.objects.create(
status=status, user=user, application=application, committee=None,
status=status,
user=user,
application=application,
committee=None,
)
status_counter += 1
for committee in application.committees.all():
Expand Down
9 changes: 6 additions & 3 deletions backend/clubs/management/commands/send_emails.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,14 +202,17 @@ def handle(self, *args, **kwargs):

if action == "update_status_reminder":
wc_badge = Badge.objects.filter(
label="Wharton Council", purpose="org",
label="Wharton Council",
purpose="org",
).first()
clubs = Club.objects.filter(badges__in=[wc_badge])
payloads = []
for club in clubs:
emails = (
Membership.objects.filter(
role__lte=Membership.ROLE_OFFICER, club__active=True, club=club,
role__lte=Membership.ROLE_OFFICER,
club__active=True,
club=club,
)
.values_list("person__email", flat=True)
.distinct()
Expand All @@ -220,7 +223,7 @@ def handle(self, *args, **kwargs):
payloads.append((applications_url, emails))
if test_email is not None:
payloads = [(payloads[0][0], [test_email])]
for (applications_url, emails) in payloads:
for applications_url, emails in payloads:
for email in emails:
if not dry_run:
template = "update_status_reminder"
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0001_initial.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@


class Migration(migrations.Migration):

initial = True

dependencies = [migrations.swappable_dependency(settings.AUTH_USER_MODEL)]
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0002_auto_20190926_1641.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@


class Migration(migrations.Migration):

dependencies = [("clubs", "0001_initial")]

operations = [
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0003_auto_20191025_2102.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@


class Migration(migrations.Migration):

dependencies = [("clubs", "0002_auto_20190926_1641")]

operations = [
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0003_auto_20191025_2120.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@


class Migration(migrations.Migration):

dependencies = [("clubs", "0002_auto_20190926_1641")]

operations = [
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0004_merge_20191025_2143.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@


class Migration(migrations.Migration):

dependencies = [
("clubs", "0003_auto_20191025_2102"),
("clubs", "0003_auto_20191025_2120"),
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0005_auto_20191025_2218.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@


class Migration(migrations.Migration):

dependencies = [("clubs", "0004_merge_20191025_2143")]

operations = [
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0006_profile.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@


class Migration(migrations.Migration):

dependencies = [
("auth", "0011_update_proxy_permissions"),
("clubs", "0005_auto_20191025_2218"),
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0007_auto_20191103_1843.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ def reverse_func(apps, schema_editor):


class Migration(migrations.Migration):

dependencies = [("clubs", "0006_profile")]

operations = [migrations.RunPython(forwards_func, reverse_func)]
1 change: 0 additions & 1 deletion backend/clubs/migrations/0008_auto_20191111_1835.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@


class Migration(migrations.Migration):

dependencies = [("clubs", "0007_auto_20191103_1843")]

operations = [
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0008_subscribe.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@


class Migration(migrations.Migration):

dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
("clubs", "0007_auto_20191103_1843"),
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0009_merge_20191206_2311.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@


class Migration(migrations.Migration):

dependencies = [("clubs", "0008_subscribe"), ("clubs", "0008_auto_20191111_1835")]

operations = []
1 change: 0 additions & 1 deletion backend/clubs/migrations/0010_note_notetag.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@


class Migration(migrations.Migration):

dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
("clubs", "0009_merge_20191206_2311"),
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0011_auto_20200103_1615.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@


class Migration(migrations.Migration):

dependencies = [("clubs", "0010_note_notetag")]

operations = [
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0012_testimonial.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@


class Migration(migrations.Migration):

dependencies = [("clubs", "0011_auto_20200103_1615")]

operations = [
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0013_auto_20200112_2104.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@


class Migration(migrations.Migration):

dependencies = [("clubs", "0012_testimonial")]

operations = [
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0014_club_youtube.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@


class Migration(migrations.Migration):

dependencies = [
("clubs", "0013_auto_20200112_2104"),
]
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0015_auto_20200119_1422.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@


class Migration(migrations.Migration):

dependencies = [
("clubs", "0014_club_youtube"),
]
Expand Down
5 changes: 3 additions & 2 deletions backend/clubs/migrations/0016_auto_20200128_2024.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,14 @@


class Migration(migrations.Migration):

dependencies = [
("clubs", "0015_auto_20200119_1422"),
]

operations = [
migrations.AlterField(
model_name="event", name="description", field=models.TextField(blank=True),
model_name="event",
name="description",
field=models.TextField(blank=True),
),
]
1 change: 0 additions & 1 deletion backend/clubs/migrations/0017_joinrequest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@


class Migration(migrations.Migration):

dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
("clubs", "0016_auto_20200128_2024"),
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0017_report.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@


class Migration(migrations.Migration):

dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
("clubs", "0016_auto_20200128_2024"),
Expand Down
6 changes: 4 additions & 2 deletions backend/clubs/migrations/0018_auto_20200207_1737.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,14 @@


class Migration(migrations.Migration):

dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
("clubs", "0017_joinrequest"),
]

operations = [
migrations.RenameModel(old_name="JoinRequest", new_name="MembershipRequest",),
migrations.RenameModel(
old_name="JoinRequest",
new_name="MembershipRequest",
),
]
1 change: 0 additions & 1 deletion backend/clubs/migrations/0019_merge_20200228_1615.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@


class Migration(migrations.Migration):

dependencies = [
("clubs", "0018_auto_20200207_1737"),
("clubs", "0017_report"),
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0020_auto_20200228_1651.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@


class Migration(migrations.Migration):

dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
("clubs", "0019_merge_20200228_1615"),
Expand Down
5 changes: 3 additions & 2 deletions backend/clubs/migrations/0021_auto_20200307_1913.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,14 @@


class Migration(migrations.Migration):

dependencies = [
("clubs", "0020_auto_20200228_1651"),
]

operations = [
migrations.AlterField(
model_name="event", name="code", field=models.SlugField(max_length=255),
model_name="event",
name="code",
field=models.SlugField(max_length=255),
),
]
1 change: 0 additions & 1 deletion backend/clubs/migrations/0022_auto_20200308_1116.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@


class Migration(migrations.Migration):

dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
("clubs", "0021_auto_20200307_1913"),
Expand Down
1 change: 0 additions & 1 deletion backend/clubs/migrations/0023_questionanswer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@


class Migration(migrations.Migration):

dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
("clubs", "0022_auto_20200308_1116"),
Expand Down
Loading
Loading