This repository has been archived by the owner on Aug 4, 2023. It is now read-only.
v1.3.6
·
124 commits
to refs/heads/main
since this release
- Filter out the extra fields with no additional info in Cleveland Museum metadata (#851) @satya-vinay
New Features
- Add reingestion DAG for Phylopic (#830) @stacimc
- Add reingestion workflow for Metropolitan (#819) @stacimc
Improvements
- Unify pre-commit config across repos (#867) @dhruvkb
- Remove unused arguments in MediaStore (#862) @bengreeley
- Remove legacy provider DAG logic (#849) @stacimc
- Refactor WordPress to use
ProviderDataIngester
(#835) @AetherUnbound - Refactor Europeana to use
ProviderDataIngester
(#821) @sarayourfriend - Refactor Rawpixel to use
ProviderDataIngester
(#795) @AetherUnbound - Refactor Flickr to use ProviderDataIngester (#809) @stacimc
- Respect ingestion limit in process_batch (#818) @stacimc
- Refactor Phylopic to use ProviderDataIngester (#747) @AetherUnbound
- Add docs and template ProviderDataIngester (#790) @stacimc
- Refactor NYPL script to use ProviderDataIngester class (#630) @obulat
- Allow default retry count to be determined by environment variable (#806) @Pmeet
- Use English for SMK results (#807) @zackkrida
- Respect the ingestion limit if ingest_records is called multiple times (#804) @stacimc
- Retire TSV loading workflow (#789) @AetherUnbound
Internal Improvements
- Bump apache-airflow[amazon,http,postgres] from 2.4.1 to 2.4.2 (#842) @dependabot
- Unify pre-commit config across repos (#867) @dhruvkb
- Adding pyupgrade pre-commit hook (#866) @aqeelat
- Bump docker/login-action from 1 to 2 (#840) @dependabot
- Unpin pytest-xdist (#843) @dependabot
- Bump alex-page/github-project-automation-plus from 0.8.1 to 0.8.2 (#841) @dependabot
- Bump actions/download-artifact from 2 to 3 (#839) @dependabot
- Make dependabot update github actions dependencies (#832) @krysal
- Refactor Smithsonian Museum to use
ProviderDataIngester
class (#812) @krysal - 🔄 Synced file(s) with WordPress/openverse (#816) @openverse-bot
- Add condition to not notify slack when don't have files in dags folder (#792) @ilitotor
- Allow default retry count to be determined by environment variable (#806) @Pmeet
- Standardize Airflow Variable names to uppercase (#801) @davcortez
- 🔄 Synced file(s) with WordPress/openverse (#814) @openverse-bot
- 🔄 Synced file(s) with WordPress/openverse (#802) @openverse-bot
Bug Fixes
- Override
get_should_continue
for science museum ingester (#868) @aqeelat - Science Museum: Handle unrecognized licenses (#850) @AetherUnbound
- Fix Europeana success test, activate reingestion workflows (#848) @AetherUnbound
- Retire Thingiverse, etlMods, commoncrawl ETL (#833) @AetherUnbound
- Restore the old dag_id for wikimedia_reingestion_workflow (#837) @stacimc
- 🔄 Synced file(s) with WordPress/openverse (#816) @openverse-bot
- Standardize Airflow Variable names to uppercase (#801) @davcortez
- 🔄 Synced file(s) with WordPress/openverse (#814) @openverse-bot
- 🔄 Synced file(s) with WordPress/openverse (#802) @openverse-bot
- Made improvements to
CONTRIBUTING.md
(#791) @kavyabhat02
Credits
Thanks to @AetherUnbound, @Pmeet, @aqeelat, @bengreeley, @davcortez, @dependabot, @dependabot[bot], @dhruvkb, @ilitotor, @kavyabhat02, @krysal, @obulat, @openverse-bot, @sarayourfriend, @satya-vinay, @stacimc and @zackkrida for their contributions!