Skip to content

Commit

Permalink
Fix Netlify build timeouts (astronomer#1294)
Browse files Browse the repository at this point in the history
Netlify jobs are failing during the "Initialization" step as seen in:
https://app.netlify.com/sites/sunny-pastelito-5ecb04/deploys/67225eeb6eaeb400080afce5

With the error:
```
5:01:30 PM: Failed during stage 'Install dependencies': Command did not finish within the time limit
5:01:30 PM: Execution timed out after 17m59.815960796s
5:01:30 PM: Execution timed out after 17m59.917466353s
5:01:30 PM: Failing build: Failed to install dependencies
```

Dependency resolution is a pain. After confirming the dependencies that
were taking longer, I pinned several of them, based on the constraints
file:
https://github.com/apache/airflow/blob/constraints-2.10.0/constraints-3.10.txt

This seems to have reduced the installation of the docs dependencies
significantly:

Locally:
```
time pip install -r requirements.txt
42.67s
```

In Netlify, the Initialization step was reduced to less than 3 min:

https://app.netlify.com/sites/sunny-pastelito-5ecb04/deploys/672364e906837e00087c53ac
  • Loading branch information
tatiana authored Oct 31, 2024
1 parent ae94975 commit 42a397f
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 9 deletions.
19 changes: 11 additions & 8 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,13 +1,16 @@
aenum
apache-airflow
apache-airflow-providers-amazon[s3fs]>=3.0.0
apache-airflow-providers-cncf-kubernetes>=5.1.1
apache-airflow-providers-google
apache-airflow-providers-microsoft-azure
google-re2==1.1
apache-airflow==2.10.0
PyAthena==3.9.0
redshift-connector==2.1.3
fsspec==2024.6.1
boto3==1.34.131
aiobotocore==2.13.2
apache-airflow-providers-amazon[s3fs]==8.27.0
apache-airflow-providers-cncf-kubernetes==8.3.4
apache-airflow-providers-google==10.21.1
apache-airflow-providers-microsoft-azure==10.3.0
apache-airflow-providers-openlineage==1.10.0
msgpack
openlineage-airflow
pydantic
pydata-sphinx-theme
sphinx
sphinx-autoapi
Expand Down
2 changes: 1 addition & 1 deletion netlify.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

# Default build command.
command = "make dirhtml"
environment = { PYTHON_VERSION = "3.8" }
environment = { PYTHON_VERSION = "3.10" }

# Production context: all deploys from the Production branch
# set in your site’s Branches settings in the UI will inherit
Expand Down

0 comments on commit 42a397f

Please sign in to comment.