From 42a397fb40ff537c74bb6f596b4936815b14abbb Mon Sep 17 00:00:00 2001 From: Tatiana Al-Chueyr Date: Thu, 31 Oct 2024 12:55:22 +0000 Subject: [PATCH] Fix Netlify build timeouts (#1294) Netlify jobs are failing during the "Initialization" step as seen in: https://app.netlify.com/sites/sunny-pastelito-5ecb04/deploys/67225eeb6eaeb400080afce5 With the error: ``` 5:01:30 PM: Failed during stage 'Install dependencies': Command did not finish within the time limit 5:01:30 PM: Execution timed out after 17m59.815960796s 5:01:30 PM: Execution timed out after 17m59.917466353s 5:01:30 PM: Failing build: Failed to install dependencies ``` Dependency resolution is a pain. After confirming the dependencies that were taking longer, I pinned several of them, based on the constraints file: https://github.com/apache/airflow/blob/constraints-2.10.0/constraints-3.10.txt This seems to have reduced the installation of the docs dependencies significantly: Locally: ``` time pip install -r requirements.txt 42.67s ``` In Netlify, the Initialization step was reduced to less than 3 min: https://app.netlify.com/sites/sunny-pastelito-5ecb04/deploys/672364e906837e00087c53ac --- docs/requirements.txt | 19 +++++++++++-------- netlify.toml | 2 +- 2 files changed, 12 insertions(+), 9 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 70481fe1c..d744fe0d3 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -1,13 +1,16 @@ aenum -apache-airflow -apache-airflow-providers-amazon[s3fs]>=3.0.0 -apache-airflow-providers-cncf-kubernetes>=5.1.1 -apache-airflow-providers-google -apache-airflow-providers-microsoft-azure -google-re2==1.1 +apache-airflow==2.10.0 +PyAthena==3.9.0 +redshift-connector==2.1.3 +fsspec==2024.6.1 +boto3==1.34.131 +aiobotocore==2.13.2 +apache-airflow-providers-amazon[s3fs]==8.27.0 +apache-airflow-providers-cncf-kubernetes==8.3.4 +apache-airflow-providers-google==10.21.1 +apache-airflow-providers-microsoft-azure==10.3.0 +apache-airflow-providers-openlineage==1.10.0 msgpack -openlineage-airflow -pydantic pydata-sphinx-theme sphinx sphinx-autoapi diff --git a/netlify.toml b/netlify.toml index 18bfb5ce4..bbf0ed645 100644 --- a/netlify.toml +++ b/netlify.toml @@ -12,7 +12,7 @@ # Default build command. command = "make dirhtml" - environment = { PYTHON_VERSION = "3.8" } + environment = { PYTHON_VERSION = "3.10" } # Production context: all deploys from the Production branch # set in your site’s Branches settings in the UI will inherit