Skip to content

Commit

Permalink
Merge pull request #47 from nasa-nccs-hpda/develop
Browse files Browse the repository at this point in the history
Develop
  • Loading branch information
jordancaraballo authored Dec 20, 2023
2 parents b9a01e5 + 07b4333 commit 221d61b
Show file tree
Hide file tree
Showing 3 changed files with 51 additions and 42 deletions.
63 changes: 23 additions & 40 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@ for high performance and commodity base environments.
- `Background`_
- `Getting Started`_
- `Infrastructure`_
- `Package Structure`_
- `Data Locations where this Workflow has been Validated`_
- `Development Pipeline Details`_
- `Authors`_
Expand Down Expand Up @@ -129,13 +128,15 @@ a singularity exec command with options from both Singularity and the cloud mask
application.

Singularity options:
- '-B': mounts a filesystem from the host into the container
- '--nv': mount container binaries/devices

* '-B': mounts a filesystem from the host into the container
* '--nv': mount container binaries/devices

vhr_cloumask_cli options:
- '-r': list of regex strings to find geotiff files to predict from
- '-o': output directory to store cloud masks
- '-s': pipeline step, to generate masks only we want to predict

* '-r': list of regex strings to find geotiff files to predict from
* '-o': output directory to store cloud masks
* '-s': pipeline step, to generate masks only we want to predict

.. code:: python
Expand Down Expand Up @@ -164,24 +165,6 @@ inference of imagery. If no GPU is available, the process will continue as expec
slowdown. There are no minimum system memory requirements given the sliding window procedures
implemented in the inference process.

Package Structure
====================

.. code:: python
├── archives <- Legacy code stored to historical reference
├── docs <- Default documentation for working with this project
├── images <- Store project images
├── notebooks <- Jupyter notebooks
├── examples <- Examples for utilizing the library
├── requirements <- Requirements for installing the dependencies
├── scripts <- Utility scripts for analysis
├── vhr_cloudmask <- Library source code
├── README.md <- The top-level README for developers using this project
├── CHANGELOG.md <- Releases documentation
├── LICENSE <- License documentation
└── setup.py <- Script to install library

Data Locations where this Workflow has been Validated
========================================================

Expand Down Expand Up @@ -230,15 +213,15 @@ the regex to the files to predict, or the output directory, manually specify the
Authors
====================

- Jordan Alexis Caraballo-Vega, jordan.a.caraballo-vega@nasa.gov
- Caleb S. Spradlin, caleb.s.spradlin@nasa.gov
- Margaret Wooten, margaret.wooten@nasa.gov
* Jordan Alexis Caraballo-Vega, jordan.a.caraballo-vega@nasa.gov
* Caleb S. Spradlin, caleb.s.spradlin@nasa.gov
* Margaret Wooten, margaret.wooten@nasa.gov

Contributors
====================

- Andrew Weis, aweis1998@icloud.com
- Brian Lee, brianlee52@bren.ucsb.edu
* Andrew Weis, aweis1998@icloud.com
* Brian Lee, brianlee52@bren.ucsb.edu

Contributing
====================
Expand All @@ -255,9 +238,10 @@ Report Bugs
Report bugs at https://github.com/nasa-nccs-hpda/vhr-cloudmask/issues.

If you are reporting a bug, please include:
- Your operating system name and version.
- Any details about your local setup that might be helpful in troubleshooting.
- Detailed steps to reproduce the bug.

* Your operating system name and version.
* Any details about your local setup that might be helpful in troubleshooting.
* Detailed steps to reproduce the bug.

Fix Bugs
-------------
Expand All @@ -283,9 +267,10 @@ Submit Feedback
The best way to send feedback is to file an issue at https://github.com/nasa-nccs-hpda/vhr-cloudmask/issues.

If you are proposing a feature:
- Explain in detail how it would work.
- Keep the scope as narrow as possible, to make it easier to implement.
- Remember that this is a volunteer-driven project, and that contributions are welcome :)

* Explain in detail how it would work.
* Keep the scope as narrow as possible, to make it easier to implement.
* Remember that this is a volunteer-driven project, and that contributions are welcome :)

Citations
============
Expand All @@ -302,8 +287,6 @@ If you find this code or methodology useful, please consider citing the followin
References
============

[1] Raschka, S., Patterson, J., & Nolet, C. (2020). Machine learning in python: Main developments and technology trends in data science, machine learning, and artificial intelligence. Information, 11(4), 193.

[2] Paszke, Adam; Gross, Sam; Chintala, Soumith; Chanan, Gregory; et all, PyTorch, (2016), GitHub repository, <https://github.com/pytorch/pytorch>. Accessed 13 February 2020.

[3] Caraballo-Vega, J., Carroll, M., Li, J., & Duffy, D. (2021, December). Towards Scalable & GPU Accelerated Earth Science Imagery Processing: An AI/ML Case Study. In AGU Fall Meeting 2021. AGU.
* Raschka, S., Patterson, J., & Nolet, C. (2020). Machine learning in python: Main developments and technology trends in data science, machine learning, and artificial intelligence. Information, 11(4), 193.
* Paszke, Adam; Gross, Sam; Chintala, Soumith; Chanan, Gregory; et all, PyTorch, (2016), GitHub repository, <https://github.com/pytorch/pytorch>. Accessed 13 February 2020.
* Caraballo-Vega, J., Carroll, M., Li, J., & Duffy, D. (2021, December). Towards Scalable & GPU Accelerated Earth Science Imagery Processing: An AI/ML Case Study. In AGU Fall Meeting 2021. AGU.
2 changes: 1 addition & 1 deletion vhr_cloudmask/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
__author__ = """Jordan A. Caraballo-Vega"""
__email__ = "jordan.a.caraballo-vega@nasa.gov"
__version__ = "1.1.0"
__version__ = "1.2.1"
28 changes: 27 additions & 1 deletion vhr_cloudmask/model/pipelines/cloudmask_cnn_pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,25 @@
# class CloudMaskPipeline
# -----------------------------------------------------------------------------
class CloudMaskPipeline(CNNSegmentation):

"""This is a conceptual class representation of a CNN Segmentation
TensorFlow pipeline. It is essentially an extended combination of the
:class:`tensorflow_caney.model.pipelines.cnn_segmentation.CNNSegmentation`.
:param logger: A logger device
:type logger: str
:param conf: Configuration device
:type conf: omegaconf.OmegeConf object
:param data_csv: CSV filename with data files for training
:type data_csv: str
:param experiment_name: Experiment name description
:type experiment_name: str
:param images_dir: Directory to store training images
:type images_dir: str
:param labels_dir: Directory to store training labels
:type labels_dir: str
:param model_dir: Directory to store trained models
:type model_dir: str
"""
# -------------------------------------------------------------------------
# __init__
# -------------------------------------------------------------------------
Expand All @@ -43,6 +61,8 @@ def __init__(
default_config: str = 'templates/cloudmask_default.yaml',
logger=None
):
"""Constructor method
"""

# Set logger
self.logger = logger if logger is not None else self._set_logger()
Expand Down Expand Up @@ -115,6 +135,12 @@ def __init__(
# predict
# -------------------------------------------------------------------------
def predict(self) -> None:
"""This will perform inference on a list of GeoTIFF files provided
as a list of regexes from the CLI.
:return: None, outputs GeoTIFF cloudmask files to disk.
:rtype: None
"""

logging.info('Starting prediction stage')

Expand Down

0 comments on commit 221d61b

Please sign in to comment.