Update database creation and add GitHub action to summarise database changes #56

znichollscr · 2024-07-29T07:48:25Z

Database creation now includes adding all known source IDs as registered if they're not already in the database. There are also entries for the datasets discussed in #11 hacked back in. These hacks can be removed once we have actual data.

~~Draft for now because I want to test out adding a GitHub action to create a comment showing the changes to the database.~~

Also adds a GitHub action to add a comment to each PR documenting the changes to the database introduced by that PR

znichollscr · 2024-07-29T07:48:51Z

@durack1 FYI. I'll ping you once I've finished with this GitHub action and we're ready for review

znichollscr · 2024-07-29T11:11:23Z

Alrighty @durack1, all ready for review now

github-actions · 2024-07-29T11:14:51Z

Changes to the database between commit 25498c0 and the 'main' branch

Overview

	operation	source_id,tracking_id
0	create	PCMDI-AMIP-OI2p1-1-0,None
1	create	DRES-CMIP-BB4CMIP7-1-0,None
2	create	CR-CMIP-0-2-0,None
3	create	SOLARIS-HEPPA-CMIP-4-1,None
4	create	PCMDI-AMIP-ERSST5-1-0,None
5	create	PCMDI-AMIP-Had1p1-1-0,None

Overview, split by source ID

Source ID: CR-CMIP-0-2-0

	operation	source_id,tracking_id
2	create	CR-CMIP-0-2-0,None

Source ID: DRES-CMIP-BB4CMIP7-1-0

	operation	source_id,tracking_id
1	create	DRES-CMIP-BB4CMIP7-1-0,None

Source ID: PCMDI-AMIP-ERSST5-1-0

	operation	source_id,tracking_id
4	create	PCMDI-AMIP-ERSST5-1-0,None

Source ID: PCMDI-AMIP-Had1p1-1-0

	operation	source_id,tracking_id
5	create	PCMDI-AMIP-Had1p1-1-0,None

Source ID: PCMDI-AMIP-OI2p1-1-0

	operation	source_id,tracking_id
0	create	PCMDI-AMIP-OI2p1-1-0,None

Source ID: SOLARIS-HEPPA-CMIP-4-1

	operation	source_id,tracking_id
3	create	SOLARIS-HEPPA-CMIP-4-1,None

Detailed changes, split by source ID

Source ID: CR-CMIP-0-2-0

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": "input4MIPs",
    "comment": null,
    "comment_post_publication": null,
    "contact": "zebedee.nicholls@climate-resource.com;malte.meinshausen@climate-resource.com",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "http://www.tbd.invalid",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "CR",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": null,
    "source_id": "CR-CMIP-0-2-0",
    "source_version": "0.2.0",
    "target_mip": null,
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

Source ID: DRES-CMIP-BB4CMIP7-1-0

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": "input4MIPs",
    "comment": null,
    "comment_post_publication": null,
    "contact": "Margreet.vanMarle@Deltares.nl;Guido.vanderWerf@wur.nl",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "http://www.globalfiredata.org",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "DRES",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": null,
    "source_id": "DRES-CMIP-BB4CMIP7-1-0",
    "source_version": "1.0",
    "target_mip": null,
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

Source ID: PCMDI-AMIP-ERSST5-1-0

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": null,
    "comment": "Prototype dataset for the evaluation of SST forcing uncertainty over the satellite era - not for production use in any simulations, including CMIP7",
    "comment_post_publication": null,
    "contact": "zelinka1@llnl.gov; durack1@llnl.gov",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "https://pcmdi.llnl.gov/mips/amip",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "PCMDI",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": "PCMDI-AMIP ERSST5 1.0: SST based on NOAA ERSST 5.0",
    "source_id": "PCMDI-AMIP-ERSST5-1-0",
    "source_version": "1.0",
    "target_mip": "Prototype",
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

Source ID: PCMDI-AMIP-Had1p1-1-0

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": null,
    "comment": "Prototype dataset for the evaluation of SST forcing uncertainty over the satellite era - not for production use in any simulations, including CMIP7",
    "comment_post_publication": null,
    "contact": "zelinka1@llnl.gov; durack1@llnl.gov",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "https://pcmdi.llnl.gov/mips/amip",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "PCMDI",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": "PCMDI-AMIP Had-1.1 1.0: SST based on UK MetOffice HadISST 1.1",
    "source_id": "PCMDI-AMIP-Had1p1-1-0",
    "source_version": "1.0",
    "target_mip": "Prototype",
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

Source ID: PCMDI-AMIP-OI2p1-1-0

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": null,
    "comment": "Prototype dataset for the evaluation of SST forcing uncertainty over the satellite era - not for production use in any simulations, including CMIP7",
    "comment_post_publication": null,
    "contact": "zelinka1@llnl.gov; durack1@llnl.gov",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "https://pcmdi.llnl.gov/mips/amip",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "PCMDI",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": "PCMDI-AMIP OI-2.1 1.0: SST based on NOAA NCEP OI2.1",
    "source_id": "PCMDI-AMIP-OI2p1-1-0",
    "source_version": "1.0",
    "target_mip": "Prototype",
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

Source ID: SOLARIS-HEPPA-CMIP-4-1

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": "input4MIPs",
    "comment": null,
    "comment_post_publication": null,
    "contact": "bernd@iaa.es",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "http://solarisheppa.geomar.de/cmip7",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "SOLARIS-HEPPA",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": null,
    "source_id": "SOLARIS-HEPPA-CMIP-4-1",
    "source_version": "4.1",
    "target_mip": "CMIP",
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

docs/input4MIPs_datasets_CMIP6Plus.html

docs/input4MIPs_files_CMIP6Plus.html

durack1 · 2024-07-29T22:52:24Z

@znichollscr I am about to run away on leave for a couple of days, so you just keep tweaking and merging updates as you make them, and I'll circle back when I am back from leave.

We have blasted away the input4MIPs_CVs.json file (master file) that would be necessary for anyone (me) using CMOR to write their data, and that will need to be remedied for the PCMDI data to be produced - as an example of what would need to be produced, you can see this PCMDI/input4MIPs-cmor-tables/Tables/input4MIPs_CV.json the old/deprecated repo where the PCMDI-AMIP-1-1-9 (and earlier) and the MRI-JRA55-do-1-6-0 (and earlier) data was generated from - we may only need to generate this for data that want to use CMOR, but that's a change from before

github-actions · 2024-07-30T05:48:00Z

Changes to the database between commit 9e53c2a and the 'main' branch

Overview

	operation	source_id,tracking_id
0	create	DRES-CMIP-BB4CMIP7-1-0,None
1	create	PCMDI-AMIP-ERSST5-1-0,None
2	create	CR-CMIP-0-2-0,None
3	create	PCMDI-AMIP-Had1p1-1-0,None
4	create	PCMDI-AMIP-OI2p1-1-0,None
5	create	SOLARIS-HEPPA-CMIP-4-1,None

Overview, split by source ID

Source ID: CR-CMIP-0-2-0

	operation	source_id,tracking_id
2	create	CR-CMIP-0-2-0,None

Source ID: DRES-CMIP-BB4CMIP7-1-0

	operation	source_id,tracking_id
0	create	DRES-CMIP-BB4CMIP7-1-0,None

Source ID: PCMDI-AMIP-ERSST5-1-0

	operation	source_id,tracking_id
1	create	PCMDI-AMIP-ERSST5-1-0,None

Source ID: PCMDI-AMIP-Had1p1-1-0

	operation	source_id,tracking_id
3	create	PCMDI-AMIP-Had1p1-1-0,None

Source ID: PCMDI-AMIP-OI2p1-1-0

	operation	source_id,tracking_id
4	create	PCMDI-AMIP-OI2p1-1-0,None

Source ID: SOLARIS-HEPPA-CMIP-4-1

	operation	source_id,tracking_id
5	create	SOLARIS-HEPPA-CMIP-4-1,None

Detailed changes, split by source ID

Source ID: CR-CMIP-0-2-0

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": "input4MIPs",
    "comment": null,
    "comment_post_publication": null,
    "contact": "zebedee.nicholls@climate-resource.com;malte.meinshausen@climate-resource.com",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "http://www.tbd.invalid",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "CR",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": null,
    "source_id": "CR-CMIP-0-2-0",
    "source_version": "0.2.0",
    "target_mip": null,
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

Source ID: DRES-CMIP-BB4CMIP7-1-0

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": "input4MIPs",
    "comment": null,
    "comment_post_publication": null,
    "contact": "Margreet.vanMarle@Deltares.nl;Guido.vanderWerf@wur.nl",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "http://www.globalfiredata.org",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "DRES",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": null,
    "source_id": "DRES-CMIP-BB4CMIP7-1-0",
    "source_version": "1.0",
    "target_mip": null,
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

Source ID: PCMDI-AMIP-ERSST5-1-0

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": null,
    "comment": "Prototype dataset for the evaluation of SST forcing uncertainty over the satellite era - not for production use in any simulations, including CMIP7",
    "comment_post_publication": null,
    "contact": "zelinka1@llnl.gov; durack1@llnl.gov",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "https://pcmdi.llnl.gov/mips/amip",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "PCMDI",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": "PCMDI-AMIP ERSST5 1.0: SST based on NOAA ERSST 5.0",
    "source_id": "PCMDI-AMIP-ERSST5-1-0",
    "source_version": "1.0",
    "target_mip": "Prototype",
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

Source ID: PCMDI-AMIP-Had1p1-1-0

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": null,
    "comment": "Prototype dataset for the evaluation of SST forcing uncertainty over the satellite era - not for production use in any simulations, including CMIP7",
    "comment_post_publication": null,
    "contact": "zelinka1@llnl.gov; durack1@llnl.gov",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "https://pcmdi.llnl.gov/mips/amip",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "PCMDI",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": "PCMDI-AMIP Had-1.1 1.0: SST based on UK MetOffice HadISST 1.1",
    "source_id": "PCMDI-AMIP-Had1p1-1-0",
    "source_version": "1.0",
    "target_mip": "Prototype",
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

Source ID: PCMDI-AMIP-OI2p1-1-0

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": null,
    "comment": "Prototype dataset for the evaluation of SST forcing uncertainty over the satellite era - not for production use in any simulations, including CMIP7",
    "comment_post_publication": null,
    "contact": "zelinka1@llnl.gov; durack1@llnl.gov",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "https://pcmdi.llnl.gov/mips/amip",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "PCMDI",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": "PCMDI-AMIP OI-2.1 1.0: SST based on NOAA NCEP OI2.1",
    "source_id": "PCMDI-AMIP-OI2p1-1-0",
    "source_version": "1.0",
    "target_mip": "Prototype",
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

Source ID: SOLARIS-HEPPA-CMIP-4-1

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": "input4MIPs",
    "comment": null,
    "comment_post_publication": null,
    "contact": "bernd@iaa.es",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "http://solarisheppa.geomar.de/cmip7",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "SOLARIS-HEPPA",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": null,
    "source_id": "SOLARIS-HEPPA-CMIP-4-1",
    "source_version": "4.1",
    "target_mip": "CMIP",
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

github-actions · 2024-07-30T05:51:09Z

Changes to the database between commit 5d47870 and the 'main' branch

Overview

	operation	source_id,tracking_id
0	create	PCMDI-AMIP-Had1p1-1-0,None
1	create	DRES-CMIP-BB4CMIP7-1-0,None
2	create	PCMDI-AMIP-ERSST5-1-0,None
3	create	PCMDI-AMIP-OI2p1-1-0,None
4	create	SOLARIS-HEPPA-CMIP-4-1,None
5	create	CR-CMIP-0-2-0,None

Overview, split by source ID

Source ID: CR-CMIP-0-2-0

	operation	source_id,tracking_id
5	create	CR-CMIP-0-2-0,None

Source ID: DRES-CMIP-BB4CMIP7-1-0

	operation	source_id,tracking_id
1	create	DRES-CMIP-BB4CMIP7-1-0,None

Source ID: PCMDI-AMIP-ERSST5-1-0

	operation	source_id,tracking_id
2	create	PCMDI-AMIP-ERSST5-1-0,None

Source ID: PCMDI-AMIP-Had1p1-1-0

	operation	source_id,tracking_id
0	create	PCMDI-AMIP-Had1p1-1-0,None

Source ID: PCMDI-AMIP-OI2p1-1-0

	operation	source_id,tracking_id
3	create	PCMDI-AMIP-OI2p1-1-0,None

Source ID: SOLARIS-HEPPA-CMIP-4-1

	operation	source_id,tracking_id
4	create	SOLARIS-HEPPA-CMIP-4-1,None

Detailed changes, split by source ID

Source ID: CR-CMIP-0-2-0

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": "input4MIPs",
    "comment": null,
    "comment_post_publication": null,
    "contact": "zebedee.nicholls@climate-resource.com;malte.meinshausen@climate-resource.com",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "http://www.tbd.invalid",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "CR",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": null,
    "source_id": "CR-CMIP-0-2-0",
    "source_version": "0.2.0",
    "target_mip": null,
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

Source ID: DRES-CMIP-BB4CMIP7-1-0

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": "input4MIPs",
    "comment": null,
    "comment_post_publication": null,
    "contact": "Margreet.vanMarle@Deltares.nl;Guido.vanderWerf@wur.nl",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "http://www.globalfiredata.org",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "DRES",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": null,
    "source_id": "DRES-CMIP-BB4CMIP7-1-0",
    "source_version": "1.0",
    "target_mip": null,
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

Source ID: PCMDI-AMIP-ERSST5-1-0

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": null,
    "comment": "Prototype dataset for the evaluation of SST forcing uncertainty over the satellite era - not for production use in any simulations, including CMIP7",
    "comment_post_publication": null,
    "contact": "zelinka1@llnl.gov; durack1@llnl.gov",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "https://pcmdi.llnl.gov/mips/amip",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "PCMDI",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": "PCMDI-AMIP ERSST5 1.0: SST based on NOAA ERSST 5.0",
    "source_id": "PCMDI-AMIP-ERSST5-1-0",
    "source_version": "1.0",
    "target_mip": "Prototype",
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

Source ID: PCMDI-AMIP-Had1p1-1-0

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": null,
    "comment": "Prototype dataset for the evaluation of SST forcing uncertainty over the satellite era - not for production use in any simulations, including CMIP7",
    "comment_post_publication": null,
    "contact": "zelinka1@llnl.gov; durack1@llnl.gov",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "https://pcmdi.llnl.gov/mips/amip",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "PCMDI",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": "PCMDI-AMIP Had-1.1 1.0: SST based on UK MetOffice HadISST 1.1",
    "source_id": "PCMDI-AMIP-Had1p1-1-0",
    "source_version": "1.0",
    "target_mip": "Prototype",
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

Source ID: PCMDI-AMIP-OI2p1-1-0

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": null,
    "comment": "Prototype dataset for the evaluation of SST forcing uncertainty over the satellite era - not for production use in any simulations, including CMIP7",
    "comment_post_publication": null,
    "contact": "zelinka1@llnl.gov; durack1@llnl.gov",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "https://pcmdi.llnl.gov/mips/amip",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "PCMDI",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": "PCMDI-AMIP OI-2.1 1.0: SST based on NOAA NCEP OI2.1",
    "source_id": "PCMDI-AMIP-OI2p1-1-0",
    "source_version": "1.0",
    "target_mip": "Prototype",
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

Source ID: SOLARIS-HEPPA-CMIP-4-1

Created entry for tracking ID: None

Entry:

{
    "Conventions": null,
    "activity_id": "input4MIPs",
    "comment": null,
    "comment_post_publication": null,
    "contact": "bernd@iaa.es",
    "creation_date": null,
    "data_node": null,
    "dataset_category": null,
    "datetime_end": null,
    "datetime_start": null,
    "esgf_dataset_master_id": null,
    "filepath": null,
    "frequency": null,
    "further_info_url": "http://solarisheppa.geomar.de/cmip7",
    "grid": null,
    "grid_label": null,
    "institution": null,
    "institution_id": "SOLARIS-HEPPA",
    "latest": null,
    "license": null,
    "license_id": "CC BY 4.0",
    "mip_era": "CMIP6Plus",
    "nominal_resolution": null,
    "product": null,
    "publication_status": "registered",
    "realm": null,
    "references": null,
    "region": null,
    "replica": null,
    "sha256": null,
    "source": null,
    "source_id": "SOLARIS-HEPPA-CMIP-4-1",
    "source_version": "4.1",
    "target_mip": "CMIP",
    "time_range": null,
    "timestamp": null,
    "tracking_id": null,
    "validated_input4mips": null,
    "variable_id": null,
    "version": null,
    "xlink": null
}

znichollscr · 2024-07-30T05:53:24Z

We have blasted away the input4MIPs_CVs.json file (master file) that would be necessary for anyone (me) using CMOR to write their data, and that will need to be remedied for the PCMDI data to be produced - as an example of what would need to be produced, you can see this PCMDI/input4MIPs-cmor-tables/Tables/input4MIPs_CV.json the old/deprecated repo where the PCMDI-AMIP-1-1-9 (and earlier) and the MRI-JRA55-do-1-6-0 (and earlier) data was generated from - we may only need to generate this for data that want to use CMOR, but that's a change from before

Alrighty, made #60 for follow up

durack1

All good to me, just a couple of questions/nits

docs/input4MIPs_datasets_CMIP6Plus.html

durack1 · 2024-08-05T13:57:08Z

docs/input4MIPs_files_CMIP6Plus.html

@@ -43973,6 +43973,264 @@
      <td>20240718</td>
      <td>None</td>
    </tr>
+    <tr>


If files don't exist, I see little point in including blank entries

Can make that tweak. That means that there will be no files with a status of 'registered', I assume that's something you're happy with?

Yeah I think so, as until files appear we're not sure what we're going to get, as some timeseries might be separated into two or more files etc - there will be no sha, and just a whole bunch of null entries

Ok cool, see #79

durack1 · 2024-08-05T13:58:01Z

python-packages/input4MIPs-CVs/pyproject.toml

+]
+name = "input4MIPs-CVs"
+version = "0.1.0"  # TODO: make this consistent with everything else
+description = """Python tooling to help with the input4MIPs-CVs repository


input4MIPs_CVs (underscore not hyphen, but I don't think this is a deal breaker)

Either way is fine. The python convention is to use hyphens in the name of the distribution (which is what we have here) and underscores in the python package name (i.e. what you put in from X import Y). Apparently this is mostly for legacy reasons, but it is where we are. That's the reason that input4MIPs validation is input4mips-validation on PyPI (https://pypi.org/project/input4mips-validation/) but you do import input4mips_validation when actually calling it in Python.

znichollscr added 2 commits July 29, 2024 09:42

Update database creation

d40c211

Update html creation

7e17fa2

znichollscr changed the title ~~Update database creation~~ Do not merge: Update database creation Jul 29, 2024

znichollscr marked this pull request as ready for review July 29, 2024 09:35

Try writing test comment

9ddb3b0

znichollscr force-pushed the registration branch from baa6722 to 9ddb3b0 Compare July 29, 2024 09:35

Next step

9769eb9

PCMDI deleted a comment from github-actions bot Jul 29, 2024

znichollscr added 2 commits July 29, 2024 11:48

Add python package skeleton

96daf47

Try first steps

7a1943f

PCMDI deleted a comment from github-actions bot Jul 29, 2024

Add setup python step

17cf8ce

PCMDI deleted a comment from github-actions bot Jul 29, 2024

znichollscr added 3 commits July 29, 2024 12:56

First crack at difference script

e4c1448

Remove sensitivity to paths

8a9af32

Add more info about source of differences

248f355

PCMDI deleted a comment from github-actions bot Jul 29, 2024

znichollscr changed the title ~~Do not merge: Update database creation~~ Update database creation and add GitHub action to summarise database changes Jul 29, 2024

znichollscr added 2 commits July 29, 2024 13:12

Make action less verbose

fcaeb01

Remove redundant comment

25498c0

PCMDI deleted a comment from github-actions bot Jul 29, 2024

durack1 approved these changes Jul 29, 2024

View reviewed changes

docs/input4MIPs_datasets_CMIP6Plus.html Outdated Show resolved Hide resolved

docs/input4MIPs_files_CMIP6Plus.html Show resolved Hide resolved

Apply suggestions from code review

9e53c2a

Fix up version in docs

5d47870

znichollscr mentioned this pull request Jul 30, 2024

Create CMOR input file #60

Closed

3 tasks

znichollscr merged commit 5cc50bc into main Jul 30, 2024
1 check passed

znichollscr deleted the registration branch July 30, 2024 05:54

znichollscr mentioned this pull request Aug 5, 2024

Register prototype AMIP datasets #11

Closed

durack1 reviewed Aug 5, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update database creation and add GitHub action to summarise database changes #56

Update database creation and add GitHub action to summarise database changes #56

znichollscr commented Jul 29, 2024 •

edited

Loading

znichollscr commented Jul 29, 2024

znichollscr commented Jul 29, 2024

github-actions bot commented Jul 29, 2024

durack1 commented Jul 29, 2024

github-actions bot commented Jul 30, 2024

github-actions bot commented Jul 30, 2024

znichollscr commented Jul 30, 2024

durack1 left a comment

durack1 Aug 5, 2024

znichollscr Aug 5, 2024

durack1 Aug 5, 2024

znichollscr Aug 5, 2024

durack1 Aug 5, 2024

znichollscr Aug 5, 2024

Update database creation and add GitHub action to summarise database changes #56

Update database creation and add GitHub action to summarise database changes #56

Conversation

znichollscr commented Jul 29, 2024 • edited Loading

znichollscr commented Jul 29, 2024

znichollscr commented Jul 29, 2024

github-actions bot commented Jul 29, 2024

durack1 commented Jul 29, 2024

github-actions bot commented Jul 30, 2024

github-actions bot commented Jul 30, 2024

znichollscr commented Jul 30, 2024

durack1 left a comment

Choose a reason for hiding this comment

durack1 Aug 5, 2024

Choose a reason for hiding this comment

znichollscr Aug 5, 2024

Choose a reason for hiding this comment

durack1 Aug 5, 2024

Choose a reason for hiding this comment

znichollscr Aug 5, 2024

Choose a reason for hiding this comment

durack1 Aug 5, 2024

Choose a reason for hiding this comment

znichollscr Aug 5, 2024

Choose a reason for hiding this comment

znichollscr commented Jul 29, 2024 •

edited

Loading