Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop #23

Merged
merged 3 commits into from
Dec 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,15 @@ If you do not have access to modify the configuration file, or just need to perf
the regex to the files to predict, or the output directory, manually specify the arguments to the CLI file:

```bash
singularity exec --env PYTHONPATH="$NOBACKUP/development/tensorflow-caney:$NOBACKUP/development/vhr-cloudmask" --nv -B $NOBACKUP,/lscratch,/explore/nobackup/people,/explore/nobackup/projects /explore/nobackup/projects/ilab/containers/above-shrubs.2023.07 python /explore/nobackup/people/jacaraba/development/vhr-cloudmask/vhr_cloudmask/view/cloudmask_cnn_pipeline_cli.py -s predict -o /explore/nobackup/projects/ilab/test/vhr-cloudmask -r /explore/nobackup/projects/3sl/data/Tappan/Tappan16_WV02_20110218_M1BS_1030010008331800_data.tif
```

### Testing

```bash
singularity exec --env PYTHONPATH="$NOBACKUP/development/tensorflow-caney:$NOBACKUP/development/vhr-cloudmask" --nv -B $NOBACKUP,/lscratch,/explore/nobackup/people,/explore/nobackup/projects /explore/nobackup/projects/ilab/containers/above-shrubs.2023.07 python /explore/nobackup/people/jacaraba/development/vhr-cloudmask/vhr_cloudmask/view/cloudmask_cnn_pipeline_cli.py

#/explore/nobackup/projects/ilab/test/vhr-cloudmask
```

## Data Locations where this Workflow has been Validated
Expand Down
2 changes: 1 addition & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

sys.path.insert(0, os.path.abspath('..'))

import vhr_cloudmask
import vhr_cloudmask # noqa: E402

project = 'vhr-cloudmask'
copyright = '2022, Jordan A. Caraballo-Vega'
Expand Down
10 changes: 10 additions & 0 deletions projects/cloud_cnn/slurm/predict.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/bin/bash
#SBATCH -t05-00:00:00 -c10 --mem-per-cpu=10240 -G1 -J ethiopia --export=ALL -q ilab
module load singularity

srun -G1 -n1 singularity exec --env PYTHONPATH="$NOBACKUP/development/tensorflow-caney:$NOBACKUP/development/vhr-cloudmask" \
--nv -B $NOBACKUP,/lscratch,/explore/nobackup/people,/explore/nobackup/projects \
/explore/nobackup/projects/ilab/containers/above-shrubs.2023.07 \
python /explore/nobackup/people/jacaraba/development/vhr-cloudmask/vhr_cloudmask/view/cloudmask_cnn_pipeline_cli.py \
-c /explore/nobackup/people/jacaraba/development/vhr-cloudmask/projects/cloud_cnn/configs/production/cloud_mask_alaska_senegal_3sl_cas.yaml \
-s predict
7 changes: 7 additions & 0 deletions projects/cloud_cnn/slurm/slurm_all.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/bin/bash

for i in {1..64}
do
sbatch /explore/nobackup/people/jacaraba/development/vhr-cloudmask/projects/cloud_cnn/slurm/predict.sh;
sleep 30s
done
4 changes: 4 additions & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,7 @@ install_requires =
numpy
tqdm
localtileserver

[options.entry_points]
console_scripts =
vhr-cloudmask-cli = vhr_cloudmask.view.cloudmask_cnn_pipeline_cli:main
Empty file added tests/cloudmask_cnn/config.yaml
Empty file.
13 changes: 13 additions & 0 deletions vhr_cloudmask/model/config/cloudmask_config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
from enum import Enum
from typing import List, Optional
from omegaconf import MISSING
from dataclasses import dataclass, field
from tensorflow_caney.model.config.cnn_config import Config


@dataclass
class CloudMaskConfig(Config):

test_classes: List[str] = field(
default_factory=lambda: ['no-cloud', 'cloud', 'thin-cloud']
)
Empty file.
Loading
Loading