Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new catalog module using Melodi API #217

Open
wants to merge 31 commits into
base: master
Choose a base branch
from
Open

new catalog module using Melodi API #217

wants to merge 31 commits into from

Conversation

hadrilec
Copy link
Contributor

@hadrilec hadrilec commented Sep 7, 2024

No description provided.

@hadrilec hadrilec added help wanted Extra attention is needed WIP labels Sep 7, 2024
tgrandje and others added 26 commits September 9, 2024 21:25
* rm tempdir on _dwn_idbank_file

* rm tempfile generated by _get_capabilities

* Systemic removal of temp files inside tempdirs

Note that some tempdir may still be present as they are generated through lrh_cache. They should be empty anyway.

+ blackformatting

* ignore_errors on shutil.rmtree

* Remove unnecessary tempfile

* Remove unnecessary tempfile and update idbank ref

* Remove unnecessary tempdir from _get_geo_relation

* Remove unnecessary tempdir from _get_dataset_dimension

* Remove unnecessary tempdir from get_last_release

* Remove unnecessary tempdir from _get_insee

* Remove unnecessary tempdir from _get_dimension_values

* Clean module imports

* Remove unnecessary tempdir from _get_geo_list_simple

* Remove unnecessary tempdir from _get_dataset_list_internal

* Remove unnecessary tempdir from _get_idbank_internal_data

* Remove unused module

* Add context manager around _download_store_file and _load_data_from_schema

* add context manager around _download_store_file and _load_data_from_schema
* rewrite _download_store_file to used predefined tempdir and fix docstring
* add a TODO in _download_store_file which does NOT used the update argument
* add context manager around file for MD5 checksum

Nota: started to refactorize this as in-memory read, but had it revert because of the size of involved files (> 4Go), which would be troublesome on old machines

* Remove unnecessary tempfile for _get_capabilities

* rewrite _get_capabilities to return an io.BytesIO object
* update _get_full_list_wfs

* Clean up after yourself

* Remove unnecessary call to _create_insee_folder and tempfile from _get_file_list_internal

* Simplify _create_insee_folder and add a warning on tempfile not beeing cleaned up

* Add call to fixed tempdir folder in _import_options

* Remove comment
* changes following new API portal

* pkg version 0.1.9 + doc update

* tests update after new API portal

* news + doc update

* example nb update + image population france update
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed WIP
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants